View PM9311-UC_295976.PDF datasheet online --- IC-ON-LINE

Datasheet File OCR Text:

Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
PM9311/2/3/5 Enhanced TT1 Chip Set Enhanced TT1 Switch Fabric Datasheet
Issue 3: August 2001
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
This document is the proprietary and confidential information of PMC-Sierra Inc. Access to this information does not transfer or grant any right or license to use this intellectual property. PMC-Sierra will grant such rights only under a separate written license agreement. Any product, process or technology described in this document is subject to intellectual property rights reserved by PMC-Sierra, Inc.and are not licensed hereunder. Nothing contained herein shall be construed as conferring by implication, estoppel or otherwise any license or right under any patent or trademark of PMC-Sierra, Inc.or any third party. Except as expressly provided for herein, nothing contained herein shall be construed as conferring any license or right under any PMC-Sierra, Inc.copyright. Each individual document published by PMC-Sierra, Inc. may contain additional or other proprietary notices and/or copyright information relating to that individual document. THE DOCUMENT MAY CONTAIN TECHNICAL INACCURACIES OR TYPOGRAPHICAL ERRORS. CHANGES ARE REGULARLY MADE TO THE INFORMATION CONTAINED IN THE DOCUMENTS. CHANGES MAY OR MAY NOT BE INCLUDED IN FUTURE EDITIONS OF THE DOCUMENT. PMC-SIERRA, INC.OR ITS SUPPLIERS MAY MAKE IMPROVEMENTS AND/OR CHANGES IN THE PRODUCTS(S), PROCESS(ES), TECHNOLOGY, DESCRIPTION(S), AND/OR PROGRAM(S) DESCRIBED IN THE DOCUMENT AT ANY TIME. THE DOCUMENT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY IMPLIED WARRANTY OR MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. ETT1, Enhanced TT1, TT1, and LCS are trademarks of PMC-Sierra, Inc.
(c) 2000 PMC-Sierra, Inc. 8555 Baxter Place Burnaby BC Canada V5A 4V7 Phone: (604) 415-6000 FAX: (604) 415-6200
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Device Part Numbers
This document contains information on the ETT1TM Chip Set, available from PMC-Sierra, Inc. The following devices comprise the ETT1 Chip Set: Device Name Scheduler Crossbar Dataslice Enhanced Port Processor PMC Part Number PM9311-UC PM9312-UC PM9313-HC PM9315-HC
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Revision History
Issue Number 1 2 Issue Date March 2000 July 2000 Creation of document Added HSTL Power Dissipation values, corrected programming constraints for EPP, added OOB, JTAG and PLL blocks to device dataflow diagrams, added heat sink information to Characteristics section, corrected EPP register section, corrected Frame Format tables, added correct bit definition for EPP Interrupt register, corrected diagram in Appendix C, added send and receive procedure for control packets, added ESOBLCP register explanation, corrected conditions in Table 57, corrected Scheduler Refresh Procedure, added new drawing for Crossbar data flow, corrected token explanation, corrected Figures 36 and 37, corrected spelling and formatting errors throughout document. Removed bit 17 from Scheduler Status Register, updated BP_FIFO information in Scheduler Control and Reset Register, corrected bottom view drawings for Scheduler and Crossbar, corrected signal description section in Scheduler and Crossbar for power pins, corrected tper3, and tpl3 in AC Electrical, added memory address information in EPP registers, corrected mechanical drawings, updated state diagram in Scheduler, added information about Scheduler PLL timing, updated initialization sequence for all chips, corrected Dataslice Signal Description for ibpen0, added bit 14(13th and 14th Dataslice enable) to EPP Control register, updated TDM constraints, added Output TDM Queue Overflow to the EPP's Interrupt register, updated EPP Output Backpressure / Unbackpressure Threshold register, updated LCS2 link synchronization in appendix, modified EPP Egress Control Packet Data Format table, Modified ETT1 usage of LCS2 protocol section Details Of Change
3
August 2001
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1
Functional Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
1.1.1 1.1.2 1.1.3 1.1.4 ETT1 Switch Core Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 The Switch Core Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 The LCS Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 The OOB (Out-Of-Band) Bus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.2
Architecture and Features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.2.1 1.2.2 1.2.3 1.2.4 1.2.5 1.2.6 1.2.7 1.2.8 1.2.9 ETT1 Switch Core . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Basic Cell Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Prioritized Best-effort Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . End-to-End Flow Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TDM Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Subport Mode (2.5 Gbit/s Linecards) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . LCS Control Packets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . System Configuration Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 19 21 22 22 23 24 25 28
1.3
Prioritized Best-Effort Queue Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
1.3.1 1.3.2 1.3.3 1.3.4 1.3.5 1.3.6 1.3.7 Unicast Traffic (OC-192). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Multicast Traffic (OC-192) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Unicast Traffic (Subport Mode). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Multicast Traffic (subport mode) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Combinations of OC-192c and Quad OC-48c Linecards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The LCS Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 31 33 46 49 50 51
1.4 1.5
Explicit Backpressure From oEPP to iEPP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
1.4.1 Flow Control Crossbar Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
TDM Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
1.5.1 1.5.2 1.5.3 1.5.4 1.5.5 1.5.6 TDM Queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TDM Reservation Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TDM Frame Timing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TDM Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Configuring the TDM Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Changing the Source of the Suggested Sync. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 55 59 59 62 63
1.6
ETT1 Usage of the LCS Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
1.6.1 1.6.2 1.6.3 1.6.4 1.6.5 LCS Protocol Prepend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Use of Label and Type Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Control Packets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Use of CRC Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . LCS PHY Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 67 69 73 75
1.7
The Out-of-Band (OOB) Bus Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
1.7.1 The OOB Bus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
i
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.8
Initialization Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
1.8.1 1.8.2 1.8.3 1.8.4 Initial Power-on: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Port Board being added:. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scheduler Board being added: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Crossbar Board being added:. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 87 89 90
1.9
Fault Tolerance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
1.9.1 1.9.2 1.9.3 1.9.4 1.9.5 1.9.6 Fault Tolerance Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Soft/Transient Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Flow Control Refresh Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scheduler Refresh Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Refresh Schedulers After Modifying Registers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Hard Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 93 97 98 99 99
1.10 ETT1 Signals and Interconnections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
1.10.1 1.10.2 1.10.3 1.10.4 LVCMOS (C and O) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 HSTL (H) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 AIB Links (A). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 Live Insertion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
1.11 System Latencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
1.11.1 1.11.2 1.11.3 1.11.4 LCS Request to Grant Minimum Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 LCS Grant to Linecard Cell Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Cell latency Through an Empty System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 LCS Hole Request to Hole Grant Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
1.12 ETT1 Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
1.12.1 1.12.2 1.12.3 1.12.4 Device Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 AIB Link Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 Linecard to ETT1 Link Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 ETT1 Internal Datapath Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
2
Dataslice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
2.1 Dataslice Blocks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
2.1.1 2.1.2 OOB Interface and Control/Status Registers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 Dataslice Cell Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
2.2 2.3
8B/10B Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 Dataslice Registers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
2.3.1 2.3.2 Dataslice Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 Dataslice Register Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
2.4
Dataslice Signal Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136
ii
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
2.5
Pinout and Package Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
2.5.1 2.5.2 2.5.3 Pinout Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 Package Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 Part Number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
3
Enhanced Port Processor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
3.1 EPP Data Flows and Blocks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
3.1.1 3.1.2 3.1.3 3.1.4 3.1.5 3.1.6 3.1.7 EPP Data Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 LCS Grant Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 Scheduler Request Modulator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 Input Queue Manager. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 Output Queue Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 Output Scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 OOB Interface and Control/Status Registers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
3.2 3.3 3.4
Input Dataslice Queue Memory Allocation with EPP . . . . . . . . . . . . . . . . . . . . 162 Output Dataslice Queue Memory Allocation with EPP . . . . . . . . . . . . . . . . . . . 162 Enhanced Port Processor Registers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
3.4.1 3.4.2 Enhanced Port Processor Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 Enhanced Port Processor Register Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
3.5 3.6
Enhanced Port processor Signal Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . 209 Pinout and Package Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
3.6.1 3.6.2 3.6.3 Pinout Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 Package Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221 Part Number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
4
Crossbar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
4.1 4.2 4.3 4.4 Crossbar Blocks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225
4.1.1 OOB Interface and Control/Status Registers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
Modes Of Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 OOB Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 Crossbar Registers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
4.4.1 4.4.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 Crossbar Register Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
4.5
Crossbar Signal Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
iii
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
4.6
Pinout and Package Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
4.6.1 4.6.2 4.6.3 Pinout Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240 Package Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 Part Number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
5
Scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
5.1 Block Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
5.1.1 5.1.2 5.1.3 5.1.4 Port Block . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 Arbiter Block . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 OOB Interface and Control/Status Registers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254 TDM Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
5.2 5.3 5.4
Port State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256 Fault Tolerance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 Scheduler Registers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
5.4.1 5.4.2 5.4.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259 Scheduler Register Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261 Input Queue Memory (IQM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
5.5 5.6
Scheduler Signal Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276 Pinout and Package Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
5.6.1 5.6.2 5.6.3 Pinout Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 Package Dimensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 Part Number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
6
Characteristics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
6.1 6.2 6.3 6.4 6.5 6.6 Signal Associations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 Absolute Maximum Ratings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297 Recommended Operating Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299 DC Electrical Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302 AC Electrical Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 Timing Diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
Appendix A
General Packet/Frame Formats . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
iv
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
A.1 Frame Formats - Dataslice To and From Enhanced Port Processor. . . . . . . . 317 A.2 Frame Formats - Dataslice To and From Crossbar. . . . . . . . . . . . . . . . . . . . . . 318 A.3 Frame Formats - Enhanced Port Processor To and From Scheduler . . . . . . . 319
Appendix B
B.1 B.2 B.3 B.4
Common Pinout Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
323 323 324 325
JTAG Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Reserved Manufacturing Test Pins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Power Supply Connections. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example Power Up Sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Appendix C
C.1.1
Interfacing Details for ETT1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
Preventing Underflow of the VOQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
C.1 Compensating for Round Trip Delay Between Linecard and ETT1(LCS) . . . . 327 C.2 The 8b/10b Interface to the Dataslice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328
C.2.1 C.2.2 Link . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329 Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
C.3 Managing Cell Rates (Setting the Idle Count Register) . . . . . . . . . . . . . . . . . . 335 C.4 Avoiding Delays Due to Idle Cells . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 336
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
v
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
vi
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 1. Figure 2. Figure 3. Figure 4. Figure 5. Figure 6. Figure 7. Figure 8. Figure 9. Figure 10. Figure 11. Figure 12. Figure 13. Figure 14. Figure 15. Figure 16. Figure 17. Figure 18. Figure 19. Figure 20. Figure 21. Figure 22. Figure 23. Figure 24. Figure 25. Figure 26. Figure 27. Figure 28. Figure 29. Figure 30. Figure 31. Figure 32. Figure 33. Figure 34. Figure 35. Figure 36. Figure 37. Figure 38. Figure 39. Figure 40. Figure 41.
The Basic Components of a Switch Built Around the ETT1 Chip Set. . . . . . . . . . . . . . . . . . . ETT1 Switch Core Logical Interconnects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Two Port Configuration of a ETT1 Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Queueing and Flow Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The TDM Frame Concept . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Fully Redundant Core . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Simple Redundant Scheduler Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . LCS Cell Sliced Across Dataslices and Crossbars . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Unicast Ingress Queueing Model for One Port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ingress and Egress Queue for Multicast Traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . An ETT1 Port Operating in Subport Mode with Four OC-48c Linecards . . . . . . . . . . . . . . . . The Input Request Counters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Full Set of Counters for a Single Priority . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . All Request Counters for a Single Output Share a Single Queue . . . . . . . . . . . . . . . . . . . . . Sequence of Events at the Input Side . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Virtual Input Queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Four Priorities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scheduler Requests/Grants are Multiplexed at a Port Level . . . . . . . . . . . . . . . . . . . . . . . . . Each iEPP has a Single Virtual Output Queue for Multicast Cells . . . . . . . . . . . . . . . . . . . . . Multicast Cell Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . EITIBM structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . OITIBM structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scheduler Priorities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ETT1 Port Ingress Queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A Separate Crossbar is Used to Convey Backpressure Information to the iEPPs. . . . . . . . . The Queueing Structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ingress Port Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The oEPP Must Treat the Transferred TDM Cell as a Multicast Cell. . . . . . . . . . . . . . . . . . . Single TDM Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TDM Synchronization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Signal Flow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . One Implementation: The OOB Bus Consists of Both Local and Global Buses . . . . . . . . . . The Global Device Select Defines the Board Number and the Device on Each Board . . . . . Write Cycle - No Wait. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Write Cycle - One Wait. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Read Cycle - One Wait . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Read Cycle - Two Waits. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Interrupts are Active High and Synchronous . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Redundant Connections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Simple Non-Redundant Crossbar Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Simple Non-Redundant Scheduler Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16 18 20 22 23 26 27 29 31 33 34 35 36 38 40 43 45 46 47 47 48 49 50 51 53 55 57 58 59 60 62 77 80 82 82 83 83 84 92 93 95
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
vii
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 42. Figure 43. Figure 44. Figure 45. Figure 46. Figure 47. Figure 48. Figure 49. Figure 50. Figure 51. Figure 52. Figure 53. Figure 54. Figure 55. Figure 56. Figure 57. Figure 58. Figure 59. Figure 60. Figure 61. Figure 62. Figure 63. Figure 64. Figure 65. Figure 66. Figure 67. Figure 68. Figure 69. Figure 70. Figure 71. Figure 72. Figure 73. Figure 74. Figure 75. Figure 76. Figure 77. Figure 78. Figure 79. Figure 80. Figure 81. Figure 82.
Simple Redundant Crossbar Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Simple Redundant Scheduler Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 ETT1 Signals and Interconnects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 AIB Links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 AIB Control/Status Signals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 Request to Grant latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 ETT1 Event Latencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 Cell Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Illustrating the Flow of Cells from the Linecard CPU to the ETT1 CPU . . . . . . . . . . . . . . . . 110 Loopback Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Control Packet Exchange Between Linecard CPU and ETT1 CPU. . . . . . . . . . . . . . . . . . . 112 Testing the ETT1 Internal Datapaths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Dataslice Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 Slicing of a Data Cell into Logical Dataslices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 Dataslice CBGA Package Dimensions - Top and Side Views . . . . . . . . . . . . . . . . . . . . . . . 147 Dataslice CBGA Package Dimensions - Bottom View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 An ETT1 Port Operating in Sub-port Mode with Four OC-48c Linecards . . . . . . . . . . . . . . 152 Functional Diagram of LCS Enhanced Port Processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 Enhanced Port Processor CBGA Package Dimensions - Top and Side Views . . . . . . . . . . 221 Enhanced Port Processor (EPP) CBGA Package Dimensions - Bottom View . . . . . . . . . . 222 The Basic Dataflow Through the Crossbar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226 Crossbar CCGA Package Dimensions - Top and Side Views . . . . . . . . . . . . . . . . . . . . . . . 249 Crossbar CCGA package Dimensions - Bottom View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250 Scheduler Block Diagram. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255 Port State Machine. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256 Scheduler CCGA package Dimensions - Top and Side Views . . . . . . . . . . . . . . . . . . . . . . 288 Scheduler CCGA package Dimensions - Bottom View . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289 Setup and Hold for 3.3V-tolerant 2.5V CMOS and 2.5V CMOS (OOB Interface) . . . . . . . . 308 Rise and Fall Times for 3.3V-tolerant 2.5V CMOS and 2.5V CMOS (OOB Interface). . . . . 309 OOB Test Loading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309 Setup and Hold for 3.3V-tolerant 2.5V CMOS and 2.5V CMOS (JTAG Interface). . . . . . . . 310 Rise and Fall Times for 3.3V-tolerant 2.5V CMOS and 2.5V CMOS (JTAG Interface) . . . . 310 JTAG Test Loading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310 Setup and Hold for 3.3V-tolerant 2.5V CMOS and 2.5V CMOS (Serdes Interface) . . . . . . 311 Rise and Fall Times for 3.3V-tolerant 2.5V CMOS and 2.5V CMOS (Serdes Interface) . . . 311 Serdes Test Loading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311 Setup and Hold for 200 Mbps HSTL (EPP/DS Interface). . . . . . . . . . . . . . . . . . . . . . . . . . . 312 Rise and Fall Times for 200 Mbps HSTL (EPP/DS Interface) . . . . . . . . . . . . . . . . . . . . . . . 312 Class 1 HSTL Test Loading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313 Setup and Hold for 800 Mbps STI (AIB Interfaces) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314 Rise and Fall Times for 800 Mbps STI (AIB Interfaces)) . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
viii
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 83. Figure 84. Figure 85. Figure 86. Figure 87. Figure 88. Figure 89. Figure 90. Figure 91. Figure 92. Figure 93. Figure 94. Figure 95. Figure 96.
AIB Test Loading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Setup and Hold for 2.5V 200 Mbps PECL (CLK & SOC Signals) . . . . . . . . . . . . . . . . . . . . Clock Period and Duty Cycle for 2.5V 200 Mbps PECL (CLK Signal) . . . . . . . . . . . . . . . . . Example Noise Isolation Circuit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example Voltage Divider . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Voltage Ramp-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Time Available to the Linecard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cells are Striped Across the 12 Physical Links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The 10-bit Data Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Initial Sequence Expected at Port Card . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cells May Be Skewed Between Dataslices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Idle Counter Can Delay an Outbound Cell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Idle Counter Can Delay an Outbound Cell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . One-deep Queue for Outbound Requests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
315 316 316 325 325 326 328 329 330 331 335 337 337 339
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
ix
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
x
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 1. Table 2. Table 3. Table 4. Table 5. Table 6. Table 7. Table 8. Table 9. Table 10. Table 11. Table 12. Table 13. Table 14. Table 15. Table 16. Table 17. Table 18. Table 19. Table 20. Table 21. Table 22. Table 23. Table 24. Table 18. Table 19. Table 20. Table 21. Table 22. Table 23. Table 24. Table 25. Table 26. Table 27. Table 28. Table 35. Table 36. Table 37. Table 38. Table 39. Table 40.
Crossbar Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 TDM Reservation Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 LCS Format from Linecard to Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 Definitions of LCS Fields from Linecard to Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 LCS Format from Switch to Linecard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 Definitions of LCS Fields from Switch to Linecard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 Encoding of 4-bit Type Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Usage of the Ingress Request Label_1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 Usage of the Egress Grant Label_1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Usage of the Egress Payload Label_2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 LCS Control Packet Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 TDM Control Packet Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Request Count Control Packet Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Start/Stop Control Packet Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Fiber 1.5 Gbit/s per Channel Interface Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 Mapping Segments to 1.5 Gbit/s Channels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 OOB Control Bus Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Multicast Group Selects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 Unicast Group Selects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Refresh-sensitive Registers in the Scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Dataslice Register Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 Dataslice Signal Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 Dataslice Pinout (left side) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 Dataslice Pinout (right side) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 Dataslice Alpha Pin List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 Dataslice CBGA Mechanical Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 Linecard Request Count Maximum Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 Input Dataslice Queue Memory Allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 Output Dataslice Queue Memory Allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 EPP Egress Control Packet Data Format and Dataslice OOB Addressing . . . . . . . . . . . . 164 Enhanced Port Processor Register Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 Enhanced Port Processor Signal Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 Enhanced Port Processor Pinout (left side) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214 Enhanced Port Processor Pinout (center) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 Enhanced Port Processor Pinout (right side) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216 Enhanced Port Processor Alpha Pin List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 Enhanced Port Processor CBGA Mechanical Specifications . . . . . . . . . . . . . . . . . . . . . . . 222 Crossbar Register Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228 Crossbar Signal Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 Crossbar Pinout (left side) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240 Crossbar Pinout (center) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
xi
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 41. Table 33. Table 34. Table 35. Table 36. Table 37. Table 38. Table 39. Table 40. Table 41. Table 42. Table 43. Table 44. Table 45. Table 46. Table 47. Table 48. Table 49. Table 50. Table 51. Table 52. Table 53. Table 54. Table 55. Table 56. Table 57. Table 58. Table 59. Table 60. Table 61. Table 62. Table 63. Table 64. Table 65. Table 66. Table 67. Table 68. Table 69. Table 70. Table 71. Table 72.
Crossbar Pinout (right side) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Crossbar Alpha Pin List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Crossbar CCGA Mechanical Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Non-TDM Traffic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scheduler Register Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scheduler Signal Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scheduler Pinout (left side) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scheduler Pinout (center) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scheduler Pinout (right side) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scheduler Alpha Pin List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scheduler CCGA Mechanical Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Signal Associations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Absolute Maximum Ratings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Absolute Maximum Ratings for 3.3V-tolerant 2.5V CMOS (OOB & Serdes Interface) . . . . Absolute Maximum Ratings for 2.5V CMOS (JTAG Interface) . . . . . . . . . . . . . . . . . . . . . . Absolute Maximum Ratings for 200 Mbps HSTL (EPP/DS Interface) . . . . . . . . . . . . . . . . . Absolute Maximum Ratings for 800 Mbps STI (AIB Interface) . . . . . . . . . . . . . . . . . . . . . . Absolute Maximum Ratings for 2.5V 200 Mbps PECL (CLK and SOC Signals) . . . . . . . . . Recommended Operating Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Additional Power Design Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DC Electrical Characteristics for 3.3V-tolerant 2.5V CMOS (OOB Interface) . . . . . . . . . . DC Electrical Characteristics for 2.5V CMOS (JTAG Interface) . . . . . . . . . . . . . . . . . . . . . DC Electrical Characteristics for 3.3V-tolerant 2.5V CMOS (Serdes) . . . . . . . . . . . . . . . . DC Electrical Characteristics for 200 Mbps HSTL (EPP/DS Interface) . . . . . . . . . . . . . . . DC Electrical Characteristics for 800 Mbps STI (AIB Interfaces) . . . . . . . . . . . . . . . . . . . . DC Electrical Characteristics for 2.5V 200 Mbps PECL (CLK and SOC Signals) . . . . . . . AC Electrical Characteristics for 3.3V-tolerant 2.5V CMOS (OOB Interface) . . . . . . . . . . . AC Electrical Characteristics for 2.5V CMOS (JTAG Interface) . . . . . . . . . . . . . . . . . . . . . AC Electrical Characteristics for 3.3V-tolerant 2.5V CMOS (Serdes Interface) . . . . . . . . . Reference Clock for 200 Mbps HSTL (EPP/DS Interface) . . . . . . . . . . . . . . . . . . . . . . . . . AC Electrical Characteristics for 800 Mbps STI (AIB Interface) . . . . . . . . . . . . . . . . . . . . AC Electrical Characteristics for 2.5V 200Mbps PECL (CLK Signals) . . . . . . . . . . . . . . . . Jitter and Static Phase Offset for PLL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Request/Data from iDS Frame Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Data from oDS Frame Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Grant/Data to iDS Frame Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Control to xDS Link Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dataslice to Crossbar Frame Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Crossbar to Dataslice Frame Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Scheduler to EPP Frame Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . EPP to Scheduler Frame Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
242 243 250 257 259 276 279 280 281 282 289 293 297 297 297 297 298 298 299 301 302 302 303 303 304 304 305 305 306 306 307 307 308 317 317 317 318 318 319 319 321
xii
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 73. Table 74. Table 75.
Suggested 8b/10b Decode Map Within the Dataslice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331 Dataslice Egress Truth Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331 Programmed Token Delay vs. Inter-link Skew . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
xiii
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
xiv
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1
1.1
Functional Description
OVERVIEW
The ETT1TM Chip Set provides a crossbar-based switch core which is capable of switching cells between 32 ports with each port operating at data rates up to 10 Gbit/s. This section describes the main features of the switch core and how cells flow through a complete system that is based on the ETT1 Chip Set. This document often refers to port rates of OC-192c or OC-48c. The ETT1 Chip Set itself operates at a fixed cell rate of 25M cells per second per port and thus is unaware of the actual data rate of the attached link. So a switch might be 32 ports of OC-192c, or it could be 32 ports of 10 Gbit/s Ethernet; it is the internal cell rate that is determined by the ETT1 Chip Set, not the link technology.
1.1.1
ETT1 Switch Core Features
The ETT1 switch core provides the following features: * * * 320 Gbit/s aggregate bandwidth - up to 32 ports of 10 Gbit/s bandwidth each Each port can be configured as 4 x OC-48c or 1 x OC-192c Both port configurations support four priorities of best-effort traffic for unicast and multicast data traffic TDM support for guaranteed bandwidth and zero delay variation with 10 Mbit/s channel resolution LCSTM protocol supports a physical separation of switch core and linecards up to 200 feet (70 m) Virtual output queues to eliminate head-of-line blocking on unicast cells Internal speedup to provide near-output-queued performance Cells are transferred using a credit mechanism to avoid cell losses due to buffer overrun In-band management and control via Control Packets Out-of-band management and control via a dedicated CPU interface Optional redundancy of all shared components for fault tolerance Efficient support for multicast with cell replication performed within the switch core
* * * * * * * * *
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
15
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.1.2
The Switch Core Model
The ETT1 Chip Set is designed to provide a switch core, not a complete packet switch system. A complete switch consists of one or more switch cores, together with a number of linecards. Each linecard connects to one port in the core. The linecard includes a physical interface (fiber, co-axial cable) to a transmission system such as SONET/SDH or Ethernet. The linecard analyzes incoming cells or packets and determines the appropriate egress port and priority. The linecard contains any cell/packet queues that are needed to allow for transient congestion through the switch. The ETT1 switch core operates on fixed size cells. If the linecard transmission system uses variable length packets, or cells of a size different from those used in the core, then the linecard is responsible for performing any segmentation and reassembly that is needed. Figure 1 illustrates this generic configuration.
Figure 1. The Basic Components of a Switch Built Around the ETT1 Chip Set.
ETT1 Switch Core port 0 Linecard 0 SONET/SDH Ethernet.... ETT1 Chip Set
Linecard 31 SONET/SDH Ethernet.... port 0 OOB Bus
LCS Protocol
CPU
The ETT1 Chip Set has been designed to allow for up to 200 feet (70 meters) of physical separation between the ETT1 core and the linecard. The LCSTM(Linecard to Switch) protocol is used between the ETT1 core and the linecard to ensure lossless transfer of cells between the two entities. However, while the LCS protocol must be implemented, the physical separation is not mandatory; the linecard could reside on the same physical board as the ETT1 port devices. The switch core has two main interfaces. One is the interface between the linecard and the core port, described in the LCS Protocol section.
16
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
The second interface is between the ETT1 devices and the local CPU. The ETT1 Chip Set requires a local CPU for configuration, diagnostics and maintenance purposes. A single CPU can control a complete ETT1 core via a common Out-Of-Band (OOB) bus. All of the ETT1 devices have an interface to the OOB bus. The OOB bus is described in Section 1.1.4 "The OOB (Out-Of-Band) Bus" on page 18.
1.1.3
The LCS Protocol
The Linecard-to-Switch (LCSTM) protocol provides a simple, clearly defined interface between the linecard and the core. In this section we introduce LCS. There are two aspects to LCS: * * a per-queue, credit-based flow control protocol a physical interface
The LCS protocol provides per-queue, credit-based flow control from the ETT1 core to the linecard, which ensures that queues are not overrun. The ETT1 core has shallow (64 cells) queues in both the ingress and egress directions. These queues compensate for the latency between the linecard and the core. One way to think of these queues is simply as extensions of the queues within the linecards. The queues themselves are described further in Section 1.3 "Prioritized Best-Effort Queue Model" on page 30. The LCS protocol is asymmetrical; it uses different flow control mechanisms for the ingress and egress flows. For the ingress flow LCS uses credits to manage the flow of cells between the linecards and the ETT1 core. The core provides the linecard with a certain number of credits for each ingress queue in the core. These credits correspond to the number of cell requests that the linecard can send to the core. For each cell request that is forwarded to a given queue in the core the linecard must decrement the number of credits for that queue. The core sends a grant (which is also a new credit) to the linecard whenever the core is ready to accept a cell in response to the cell request. At some later time, which is dependent on the complete traffic load, the cell will be forwarded through the ETT1 core to the egress port. In the egress direction a linecard can send hole requests, requesting that the ETT1 core does not forward a cell for one celltime. The linecard can issue a hole request for each of the four best effort unicast or multicast priorities. If the linecard continually issued hole requests at all four priorities then the ETT1 core would not forward any best effort traffic to the linecard. The LCS protocol information is contained within an eight byte header that is added to every cell. The physical interface that has been implemented in the ETT1 Chip Set is based on a faster version of the Gigabit Ethernet Serdes interface, enabling the use of off-the-shelf parts for the physical link. This interface provides a 1.5 Gbit/s serial link that uses 8b/10b encoded data. Twelve of these links are combined to provide a single LCS link operating at 18 Gbaud, providing an effective data bandwidth that is in excess of an OC-192c link. NOTE: The LCS protocol is defined in the "LCS Protocol Specification -- Protocol Version 2", available from PMC-Sierra, Inc. This version of LCS supersedes LCS Version 1. Version 2 is first supported in the TT1 Chip Set with the Enhanced Port Processor device (also referred to as the ETT1 Chip Set) and will be supported in future PMC-Sierra products. The ETT1 implementation of the LCS protocol is described further in Section 1.6 "ETT1 Usage of the LCS Protocol" on page 64.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
17
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.1.4
The OOB (Out-Of-Band) Bus
The ETT1 Chip Set requires a local CPU to perform initialization and configuration after the Chip Set is reset or if the core configuration is changed - perhaps new ports are added or removed, for example. The OOB bus provides a simple mechanism whereby a local CPU can configure each device. Logically, the OOB bus provides a 32 bit address/data bus with read/write, valid and ready signals. The purpose of the OOB bus is for maintenance and diagnostics; the CPU is not involved in any per-cell operations.
1.2 1.2.1
ARCHITECTURE AND FEATURES ETT1 Switch Core
An ETT1 switch core consists of four types of entities: Port, Crossbar, Scheduler, and CPU/Clock. There may be one or more instances of each entity within a switch. For example, each entity might be implemented as a separate PCB, with each PCB interconnected via a single midplane PCB. Figure 2 illustrates the logical relationship between these entities.
Figure 2. ETT1 Switch Core Logical Interconnects
Crossbar Linecard Port Port Linecard
Flow Control Crossbar
Scheduler
CPU/clock
18
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Each ETT1 port is attached to one or more linecards. The port contains the shallow cell queues and implements the LCS protocol. The port and Scheduler exchange information about cells that are waiting to be forwarded through the Crossbar core. The Scheduler maintains local information on the number of cells that are waiting in all of the ingress and egress queues. It arbitrates amongst all cells in the ingress queues, and instructs all of the ports as to which cell they can forward through the Crossbar at each cell time. Two Crossbars are used. The first, referred to as simply `the Crossbar', interconnects all of the ports with all of the other ports, enabling cells to be forwarded from the ingress port queues to the egress port queues (in a different port). The Crossbar is reconfigured at every cell time to provide any non-blocking one-to-one or one-to-many mapping from input ports to output ports. Each Crossbar port receives its configuration information from its attached port; the Crossbars do not communicate directly with the Scheduler. The second Crossbar is the flow-control Crossbar. It passes output queue occupancy information from every egress port to every ingress port. The ingress ports use this information to determine when requests should and should not be made to the Scheduler. The CPU/clock provides clocks and cell boundary information to every ETT1 device. It also has a local CPU which can read and write state information in every ETT1 device via the OOB bus. The CPU/clock entity is a necessary element of the ETT1 switch core, but does not contain any of the ETT1 devices.
1.2.2
Basic Cell Flow
The ETT1 Chip Set consists of four devices. Their names (abbreviations) are: * * * * Dataslice (DS) Enhanced Port Processor (EPP) Scheduler (Sched) Crossbar (Xbar)
This section describes how cells flow through these four devices.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
19
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 3. Two Port Configuration of a ETT1 Switch
1,3 2 1,3
Input Data Slice
6 5
Crossbar
7
Output Data Slice
5
2,3
Input EPP
5 7
Flow Control Crossbar
6 7
5,7
Output EPP
4
Scheduler data cell flow control flow
4
3
6,7
Figure 3 shows a two-port configuration of a ETT1 switch. Only the ingress queues of the left hand port and the egress queues of the right hand port are shown. The port has one EPP and either six or seven DS devices. The DS contains the cell queue memory and also has the Serdes interface to the linecard. A single cell is "sliced" across all of the Dataslice devices, each of which can manage two slices. The EPP is the port controller, and determines where cells should be stored in the DS memories. Multiple Crossbar devices make up a full Crossbar. A single Scheduler device can arbitrate for the entire core. A cell traverses the switch core in the following sequence of events: 1. A cell request arrives at the ingress port, and is passed to the EPP. The EPP adds the request to any other outstanding cell requests for the same queue. 2. At some later time, the EPP issues a grant/credit to the source linecard, requesting an actual cell for a specific queue. The linecard must respond with the cell within a short period of time. 3. The cell arrives at the ingress port and the LCS header is passed to the EPP. The EPP determines the destination queue from the LCS header, and then tells the Dataslices where to store the cell (each Dataslice stores part of the cell). The EPP also informs the Scheduler that a new cell has arrived and so the Scheduler should add it to the list of cells waiting to be forwarded through the Crossbar. The EPP modifies the LCS label by replacing the destination port with the source port, so the egress port and linecard can see which port sent the cell. 4. The Scheduler arbitrates among all queued cells and sends a grant to those ports that can forward a cell. The Scheduler also sends a routing tag to each of the destination (egress) ports; this tag
20
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
tells the ports which of the many source ports will be sending it a cell. 5. The source EPP sends a read command to the Dataslices, which then reads the cell from the appropriate queue and sends it to the Crossbar. At the same time, the destination ports send the routing tag information to the Crossbar. This routing tag information is used to configure the internal connections within the Crossbar for the duration of one cell time. The cell then flows through the Crossbar from the source port to the destination port. 6. The cell arrives at the destination port, and the EPP receives the LCS header of the cell. It uses this information to decide in which egress queue the cell should be stored. If this was a multicast cell which caused the egress multicast to reach its occupancy limit, then the EPP would send a congestion notification to the Scheduler. 7. At some later time, the EPP decides to forward the cell to the linecard. The EPP sends a read command to the Dataslices which read the cell from memory and forward the cell out to the linecard. The egress EPP also sends flow control to the ingress EPP, informing it that there now exists free space in one or more of the egress EPP's output queues. Also, if the transmitted cell was a multicast cell then this may cause the egress queue to go from full to not full, in which case the EPP notifies the Scheduler that it (the EPP) can once again accept multicast cells. The above description does not account for all of the interactions that can take place between ETT1 devices, but it describes the most frequent events. In general, users do not need to be aware of the detailed interactions, however knowledge of the main information flows will assist in gaining an understanding of some of the more complicated sections.
1.2.3
Prioritized Best-effort Service
An ETT1 switch core provides two types of service. The first is a prioritized, best-effort service. The second provides guaranteed bandwidth and is described later. The best-effort service is very simple. Linecards forward best-effort cells to the ETT1 core where they will be queued. The Scheduler arbitrates among the various cells; the arbitration algorithm has the dual goals of maximizing throughput while providing fair access to all ports. If more than one cell is destined for the same egress port then the Scheduler will grant one of the cells and the others will remain in their ingress queues awaiting another round of arbitration. The service is best-effort in that the Scheduler tries its best to satisfy all queued cells, but in the case of contention then some cells will be delayed. The Scheduler supports four levels of strict priority for best effort traffic. Level 0 cells have the highest priority, and level 3 cells have the lowest priority. A level 0 cell destined for a given port will always be granted before a cell of a different priority level, in the same ingress port, that is destined for the same egress port. A `flow' is a sequence of cells from the same ingress port to the same egress port(s) at a given priority. Best-effort flows are either unicast flows (cells in the flow go to only one egress port), or multicast flows (in which case cells can go to many, even all, of the egress ports).
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
21
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.2.4
End-to-End Flow Control
The full queueing and flow control model is shown in Figure 4. NOTE: Credits or backpressure are used at every transfer point to ensure that cells cannot be lost due to lack of buffer space.
Figure 4. Queueing and Flow Control
ETT1 core
Linecard
ETT1 port
ETT1 Crossbar
ETT1 port
Linecard
Ingress queues LCS credits
Ingress queues
Egress queues hole requests
Egress queues
ETT1 backpressure
cell flow backpressure/credits
1.2.5
TDM Service
The ETT1 TDM service provides guaranteed bandwidth and zero cell delay variation. These properties, which are not available from the best-effort service, mean that the TDM service might be used to provide an ATM CBR service, for example. The ETT1 core provides the TDM service at the same time as the best effort service, and TDM cells integrate smoothly with the flow of best-effort traffic. In effect, the TDM cells appear to the Scheduler to be cells of the highest precedence, even greater than level zero best-effort multicast traffic. The TDM service operates by enabling a ETT1 port (and linecard) to reserve the crossbar fabric at some specified cell time in the future. The Scheduler is notified of this reservation and will not schedule any best-effort cells from the ingress port or to the egress port during that one cell time. Each port can make separate reservations according to whether it will send and/or receive a cell at each cell time. Several egress ports may receive the same cell from a given ingress port; therefore, the TDM service is inherently a multicast service. In order to provide a guaranteed bandwidth over a long period of time, an ingress port will want to repeat the reservations on a regular basis. To support this the ETT1 core uses an internal construct called a TDM
22
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Frame. A TDM Frame is simply a sequence of ingress and egress reservations. The length (number of cell times) of a TDM Frame is configurable up to a maximum of 1024 cells. The TDM Frame repeats after a certain fixed time. All ports are synchronized to the start of the TDM Frame, and operate cell-synchronously with respect to each other. Thus, at any cell time, every ETT1 port knows whether it has made a reservation to send or receive a TDM cell. See the application note "LCS-2 TDM Service in the ETT1 and TTX Switch Core", available from PMC-Sierra, Inc. Figure 5 illustrates the idea of a TDM Frame. The TDM Frame has N slots (N is 1024 or less) where each slot is one 40ns cell time. The TDM Frame is repeated continuously, but adjacent TDM Frames are separated by a guard band of at least 144 cells.
Figure 5. The TDM Frame Concept
40ns
1
N
One TDM Frame
Guard band
All of the ETT1 ports must be synchronized with respect to the start of the TDM Frame. This synchronization information is distributed via the Scheduler. The Scheduler can generate the synchronization pulses itself, or it can receive "Suggested Sync" pulses from a ETT1 port which can in turn receive synchronization signals from a linecard. The linecards do not need to be exactly synchronized to either the ETT1 core or the other linecards. The LCS protocol will compensate for the asynchrony between the linecard and the ETT1 core.
1.2.6
Subport Mode (2.5 Gbit/s Linecards)
An ETT1 switch core can have up to 32 ports. Each port supports a linecard bandwidth in excess of 10 Gbit/s. This bandwidth might be used by a single linecard (for example, OC-192c), or the ETT1 port can be configured to share this bandwidth among up to four ports, each of 2.5 Gbit/s. This latter mode, specifically four 2.5 Gbit/s linecards, is referred to as subport mode, or sometimes as quad OC-48c mode. A single ETT1 switch can have some ports in `normal' mode, and some in subport mode; there is no restriction. The four levels of prioritized best-effort traffic are available in all configurations. However, there are some important differences between the two modes. One difference is that the LCS header must now identify the subport associated with each cell. Two bits in the LCS label fields are used for this purpose, as described below. A second difference is that the EPP must carefully manage the output rate of each subport, so as not to overflow the buffers at the destination linecard, and this is also described below.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
23
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
A third difference is that the EPP must maintain separate LCS request counters for each of the subports. It must also maintain separate egress queues. So the number of queues can increase four-fold in order to preserve the independence of each subport. Section 1.3 "Prioritized Best-Effort Queue Model" on page 30 describes the various queuing models in great detail.
1.2.6.1
Identifying the Source Subport
The EPP manages a single physical stream of cells at 25M cells/second. In subport mode the EPP must look at each incoming cell and determine which subport has sent the cell. The LCS label field is used to achieve this. Two bits within the label (referred to in the LCS Specification as MUX bits) are used to denote the source subport, numbered 0 through 3. The MUX bits must be inserted in the LCS label before the cell arrives at the EPP. The MUX bits might be inserted at the source linecards themselves. Alternatively, they might be inserted by a four-to-one multiplexer device placed between the subport linecards and the EPP. In this latter case, the multiplexer device might not be able to re-calculate the LCS CRC. For maximum flexibility, the EPP can be configured to calculate the LCS CRC either with or without the MUX bits.
1.2.6.2
Egress Cell Rate
Within the EPP is an Output Scheduler process which is different from, and should not be confused with, the ETT1 Scheduler device. At every OC-192 cell time, the Output Scheduler looks at the egress queues and decides which cell should be forwarded to the attached egress linecard(s). In subport mode, the Output Scheduler will constrain the egress cell rate so as not to overflow any of the 2.5 Gbit/s links. It does this by operating in a strict round-robin mode, so that at any OC-192 cell time it will only try to send a cell for one of the subports. The four 2.5 Gbit/s subports are labeled as 0 through 3 ; at some cell time the Output Scheduler will only try to send a cell from the egress queues associated with subport 0. In the next time, it will only consider cells destined for subport 1, etc. If, at any cell time, there are no cells to be sent to the selected subport, then an Idle (empty) cell is sent. For example, if subports 0 and 3 are connected to linecards but subports 2 and 4 are disconnected, then the Output Scheduler will send a cell to 0, then an empty cell, then a cell to 3, then an empty cell, and then repeat the sequence. The effective cell rate transmitted to each subport will not exceed 6.25 M cells per second.
1.2.7
LCS Control Packets
The LCS protocol provides in-band control packets. These packets (cells) are distinct from normal cell traffic in that they do not pass through the fabric to an egress linecard, but are intended to cause some effect within the switch. There are two classes of Control Packets. The first class, referred to as CPU Control Packets, are exchanged between the linecard and the ETT1 CPU (via the EPP and Dataslices). The intention is that CPU Control Packets form the basic mechanism through which the linecard CPU and ETT1 CPU can exchange information. This simple mechanism is subject to cell loss, and so should be supplemented by some form of reliable transport protocol that would operate within the ETT1 CPU and the linecards. The second class, referred to as LCS Control Packets, are used to manage the link between the linecard and the ETT1 port. These LCS Control Packets can be used to start and stop the flow of cells on the link,
24
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
to provide TDM synchronization event information, and can recover from any grant/credit information that is lost if cells are corrupted in transmission.
1.2.7.1
Sending a Control Packet from the OOB to the Linecard
Before sending a CPU (OOB to linecard) control packet, the OOB must first write the control packet header and payload data into the appropriate locations in the Dataslice. (See Section 3.3 "Output Dataslice Queue Memory Allocation with EPP", Table 23, on page 164.) The header for customer-specific CPU control packets should be written into the Dataslices as shown, but the payload data is completely up to the customer. To send the CPU control packet which has been written into Dataslice queue memory, the OOB writes the ESOBLCP register with the control packet type select in bits [5:4] (see the bit-breakout), and a linecard fanout in bits [3:0]. If the port is connected to 4 subport linecards, then bits [3:0] are a subport-bitmap. If the port is connected to one OC-192c linecard, then bit 0 must be set when the OOB wishes to send a CP. When the CP has been sent to the linecard(s) indicated in bits [3:0], bits [3:0] will read back as 0. Since control packets have higher priority than any other traffic type, they will be sent immediately, unless the EPP is programmed to send only idle cells.
1.2.7.2
Sending a Control Packet From the Linecard to the OOB
The linecard sends control packets to OOB using the regular LCS request/grant/cell mechanism. A CPU (linecard to OOB) control packet must have the CPU bit set in its request label (See Section 1.1.3 "The LCS Protocol" on page 17.) When the EPP receives the cell payload for a CPU control packet, it stores the cell in the Dataslices' Input Queue memories and raises the "Received LC2OOB/CPU Control Packet from Linecard..." interrupt. (In OC-192c mode, it will always be Linecard 0.) The input queue for CPU control packets from each linecard is only 8 cells deep, so as soon as the OOB sees a "Received LC2OOB..." interrupt, it should read the appropriate "Subport* to OOB FIFO Status" register (0x80..0x8c). Bit [3:0] of that register will tell how many control packet cells are currently in the input queue; bits 6:4 will tell the offset of the head the 8-cell CPU CP input queue. That queue offset should be used to form addresses for the Dataslices' Input Queue memories. See Section 3.2 "Input Dataslice Queue Memory Allocation with EPP" on page 162, for Dataslice Input Queue memory addressing. Then the OOB should read those addresses to obtain the CPU CP payload data. When the OOB has read the CPU CP payload data, it should write the appropriate "Linecard * to OOB FIFO Status" register (any value). A write to that register, regardless of the write data, will cause the head of the queue to be dequeued, freeing up that space in the CPU CP input queue. See Section 1.6.3 "Control Packets" on page 69 for more details.
1.2.8
Redundancy
An ETT1 core can be configured with certain redundant (duplicated) elements. A fully redundant core is capable of sustaining single errors within any shared device without losing or re-ordering any cells. This section describes the main aspects of a redundant core. A complete switch may have two ETT1 cores,
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
25
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
with linecards dual-homed to both cores, and while this is a valid configuration the ETT1 core does not provide any specific support for such a configuration. Figure 6 shows a fully redundant core. The Crossbars and Scheduler are fully replicated, as are the links connecting those devices. The port devices (EPP and Dataslices) are not replicated. It is important to understand that if an EPP or Dataslice fails (or incurs a transient internal error), then data may be lost, but only within that port.
Figure 6. Fully Redundant Core
Crossbar 0
Dataslice Crossbar 1
Dataslice
EPP Flow Control Crossbar 0 Port Flow Control Crossbar 1
EPP
Port
Scheduler 0
Scheduler- 1
Fault tolerant region
A fully redundant core operates in exactly the same way as a non-redundant core. Consider the EPP: it sends new request information to both Schedulers. The two Schedulers operate synchronously, producing identical outputs, and the information (grants) received by the EPP will be the same. The Dataslices operate in exactly the same way with their two Crossbar devices.
26
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
The information carried on the links is protected by checksums. For example, if the EPP receives information with a checksum error on one link, then it simply ignores that link and uses the information from the other link. All errors are reported to the local CPU via interrupts. The redundant core is tolerant of such single errors, provided that they are detectable. In general, all errors that occur on the links or within the Scheduler or Crossbar will be detectable. It should be clear from Figure 6 that only the Dataslice and EPP have the redundant connections. An individual Crossbar or Scheduler cannot compensate for information that is received with a checksum error. The following discussion reviews how these two types of devices are affected by receiving corrupted information. If a Crossbar detects a checksum error then it simply marks the cell as being corrupted. This information is passed to the egress Dataslice which will ignore that link and use the information from the other Crossbar. The Scheduler is more complicated because it must maintain accurate state information on the occupancy of all queues as well as the backpressure status of some of the egress queues. If a Scheduler receives a checksum error then it must effectively remove itself from the system. Consider the simple configuration illustrated in Figure 7. Scheduler 0 sees a checksum error on information it receives from EPP 1. Scheduler 0 now has incorrect state information. It immediately disables all of its outbound links so that EPP 1 and EPP 2 know that Scheduler 0 should be ignored. The EPPs now only accept information from Scheduler 1.
Figure 7. Simple Redundant Scheduler Configuration
EPP 1
EPP 2
Scheduler 0
Scheduler 1
If a new Scheduler-0 board is inserted in the system, then it must have its internal state updated to match that of Scheduler-1. This is done using a sequence of actions that are controlled by the ETT1 CPU. In essence, the linecards are temporarily stopped from sending new cells, and the state within each EPP is downloaded into the Schedulers, and then the linecards are restarted. The entire process is very rapid (much less than 1ms), but does cause best-effort traffic to be suspended for a brief time.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
27
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.2.9
System Configuration Options
Most of the previous sections have assumed a particular switch configuration consisting of 32 ports of OC-192c, with 64 byte payload cells and full redundancy. The ETT1 Chip Set has four aspects that can be configured according to the user's requirements. Two of these aspects have been described: quad OC48c versus single OC-192c port, and a redundant system versus a non-redundant system. The other two aspects are described in this section. NOTE: All four aspects are orthogonal and so the choice of any one particular aspect does not limit the choices for the other three aspects.
1.2.9.1
Payload: 64 bytes or 76 bytes
The ETT1 core and LCS protocol are designed to forward fixed length cells. Each LCS cell consists of an LCS header (8 bytes) and a payload. The size of this payload can be either 64 bytes or 76 bytes. The choice of payload size is a function of the linecard traffic: 53-byte ATM cells can probably use a 64 byte payload; fragmented IP packets might obtain greater efficiency with a 76 byte payload. This flexibility in cell size is due to the way the ETT1 Chip Set slices the cells. Figure 8 shows how an LCS cell is sliced across the Dataslices and Crossbars.
28
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 8. LCS Cell Sliced Across Dataslices and Crossbars
ETT1 Port LCS Cell LCS Header (8 Bytes) Dataslice Slices 0,1 Crossbar 0 Crossbar 1 Crossbar 2 Crossbar 3
ETT1 Port
Dataslice Slices 0,1
Dataslice Slices 2,3 Payload (64 Bytes)
Dataslice Slices 2,3
Dataslice Slices 10,11
Crossbar 10 Crossbar 11 Crossbar 12 Crossbar 13
Dataslice Slices 10,11
Extra 12 bytes (6 bytes per slice)
Dataslice Slices 12,13
Dataslice Slices 12,13
The 64 byte payload cell requires six Dataslices per port and 12 Crossbars (32 port, non-redundant). The 76 byte payload cell requires seven Dataslices per port and 14 Crossbars. The EPP is capable of supporting the seventh Dataslice. Because the extra data is obtained by slicing the cell then this does not affect the clock frequency of the Chip Set. The only effect is that the link between the ETT1 port and the linecard must go up from 18 Gbaud to 21 Gbaud. This is done by making the link wider, not faster.
1.2.9.2
8, 16, or 32 ports
The ETT1 core can support a maximum of 32 ports. Smaller configurations can be supported by simply not using some of the ports. However, if the maximum system is an eight or 16 port configuration, then some
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
29
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
cost savings can be achieved by using fewer Crossbar devices. A Crossbar can be configured to be in an 8- 16- or 32-port core, as shown in Table 1.
Table 1. Crossbar Configurations
Number of OC-192c ports 1 to 8 9 to 16 17 to 32
Number of Crossbars required (non redundant) 3 + 1 Flow Control 6 + 1 Flow Control 12 + 2 Flow Control
Number of Crossbars required (redundant) 6 + 2 Flow Control 12 + 2 Flow Control 24 + 4 Flow Control
The reduction in the number of crossbars is achieved by having a single port use more than one connection on each Crossbar. In a 32 port system, each port will use one connection to each Crossbar device; in an 8 port system, each port will use four connections on each Crossbar. (This DOES NOT apply to Flow Control Crossbars; each port will always use only one connection.)
1.3
PRIORITIZED BEST-EFFORT QUEUE MODEL
The ETT1 switch core has been designed to maximize cell throughput while keeping latency small and providing "fair" throughput among all ports. The ETT1 core supports four levels of strict priority. Level 0 is the highest priority and level 3 the lowest. All conditions being equal, the core will always forward a higher priority cell before a lower priority cell. Furthermore, the core will be fair to all inputs. This fairness is very difficult to quantify and is only meaningful over time periods of many cells. Over short time periods (tens or perhaps hundreds of cells) there may be temporary unfairness between ports. A cell of a lower priority may pass a cell of a higher priority under certain conditions: 1. Since the scheduling pipeline is shorter for lower priorities, a cell of a lower priority may emerge from the switch sooner than a cell of a higher priority if the lower-priority cell is sent immediately after the higher-priority cell. 2. Hole requests for a higher priority can allow cells of a lower priority to pass cells of the higher priority. 3. If the linecard responds to LCS grants faster than required to meet the system round trip time constraint, then multicast cells of a lower priority may pass unicast cells of a higher priority, due to a (programmable) delay in the EPP's "Internal Delay Matching Adjustments" register. 4. In subport mode only: multicast output queues (VIQs) are shared by subports, so when a subport is oversubscribed for an extended period of time, blocking may result on all subports. This blocking of multicast can allow unicast cells of the same priority to pass. In this section we describe the queueing model provided by the ETT1 switch core for this prioritized best-effort traffic. We start by considering only OC-192c ports. The following sections describe how the model differs for OC-48c ports.
30
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.3.1
Unicast Traffic (OC-192)
Every ETT1 port has a separate request counter and ingress queue for each (output port, priority) combination to which it can send unicast cells. In a full configuration, an ETT1 port can send to 32 ETT1 ports (including itself), at four priorities, and so will have 128 unicast request counters and ingress queues. All of these counters and queues are independent so that the switch does not suffer from head-of-line blocking1. Figure 9 illustrates the unicast ingress queueing model for one port.
Figure 9. The Unicast Ingress Queueing Model for One Port
The ingress queues are referred to as Virtual Output Queues (VOQs). This name indicates that each queue holds cells going to just one output port. Every ETT1 port also has the same number of unicast egress queues; one queue for every (input port, priority) combination that can send cells to the port. Every port will have 128 unicast egress queues. It is not acceptable to have a single egress queue per priority, as this can lead to gross unfairness between ports. These egress unicast ports are called Virtual Input Queues (VIQs) as each queue only holds cells that have come from a single input port. A useful model is to think of a VOQ as being directly connected through the Crossbar to a single VIQ. The cell at the head of the VOQ is waiting to be scheduled in such a way that it can move to the tail of the VIQ in the egress port. The associated ingress EPP is informed whenever an egress queue is full. In this case, the ingress port will not issue any new requests to the Scheduler until sufficient cell space becomes available at the egress queue. The transfer of cells from ingress to egress queues is lossless from a queueing perspective. The best effort unicast ingress and egress queues can store up to 64 cells.
1.3.2
Multicast Traffic (OC-192)
The ETT1 core also supports multicast cells. Multicast cells are cells that must be forwarded to one or more egress ports. The LCS header will indicate that a cell is multicast, and it will also contain a multicast group identifier. The ETT1 port uses this identifier to determine the list of ports to which the cell should go. This list of ports is called the multicast fanout. Different multicast groups will probably have different fanouts. The Scheduler arbitrates among unicast and multicast cells. A multicast cell of a given priority will always have priority over a unicast cell of the same priority, all else being equal. Physical cell replication takes place within the Crossbar.
1. Head -of-line blocking is a phenomenon encountered by input queued switches in the case where a cell destined for one output port is delayed because the cell at the head of the same FIFO queue is destined for some other output which is currently congested.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
31
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
The Scheduler will not necessarily schedule all ports in a fanout at the same cell time. In other words, the same multicast cell might pass through the Crossbar on several occasions before all egress ports have received their copy of the cell. Every ETT1 port has a single ingress queue and a single egress queue for multicast traffic at each priority, as shown in Figure 10. (There are also request counters, as for unicast cells, but these are not shown.)
32
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 10. Ingress and Egress Queue for Multicast Traffic
The Scheduler maintains information on the occupancy of the multicast ingress queues, and receives backpressure/unbackpressure signals for the multicast egress queues. If an egress multicast queue is backpressured then there is a question as to what happens to the cell at the head of an ingress queue that is trying to send to that backpressured egress queue (as well as other ports). If the Scheduler blocks the ingress queue, then the system will experience head-of-line blocking for multicast cells at that priority. Every ETT1 port can select how it wants to handle multicast cells that are sent to it. For each of the four multicast egress queues, the local ETT1 CPU can either enable or disable a backpressure signal from being sent to the Scheduler. NOTE: This selection is at the egress queues, not the ingress queues. If a port is set so that multicast backpressure is disabled, then that port will never cause head-of-line blocking to occur in any ingress multicast queue that will send to it. If a port is set so that multicast backpressure is enabled, then that port can assert a backpressure signal if the egress queue gets too full. Consequently, this might cause head-of-line blocking at multicast ingress queues that send cells to this port. Therefore, head-of-line blocking is only guaranteed not to occur if all of the ports that can receive multicast cells have their multicast backpressure signals disabled. If at least one port has backpressure enabled, then head-of-line blocking might occur.
1.3.3
Unicast Traffic (Subport Mode)
The EPP can operate in one of two modes. The first mode uses the queueing model described previously, and corresponds to a single channel of approximately 10 Gbit/s. The second mode is called subport mode. In subport mode an EPP can manage four channels of 2.5 Gbit/s each. The physical interface at the Dataslice is always a single channel; in subport mode the four 2.5 Gbit/s channels are time multiplexed onto the single 10 Gbit/s channel. More precisely, given that the ETT1 Chip Set always operates at 25M cells per second, the subport mode correspond to four channels each of up to 6.25M cells per second. Figure 11. shows the block diagram of an ETT1 port card. The six (or seven) Dataslices provide a single interface. A separate MUX device is required to manage the four OC-48c channels and merge them so they appear as a single channel to the EPP/DS.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
33
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 11. An ETT1 Port Operating in Subport Mode with Four OC-48c Linecards
ETT1 Port board OC-48c 0 Dataslice 0/1 Dataslice 0/1 Dataslice 0/1 Dataslice 0/1 Dataslice 0/1 Dataslice 0/1 25M cell/s interface OC-48c 3 Scheduler EPP Flow Control Crossbar Crossbar Crossbar Crossbar Crossbar Crossbar Crossbar MUX
ETT1 Port board
OC-48c 1
OC-48c 2
The EPP can support all four best-effort priorities when operating in subport mode. This is the primary feature that differentiates the EPP from the PP. Furthermore, the EPP can operate in a mixed mode switch, so an ETT1 switch can have some ports operating in subport mode (quad OC-48c), while other ports operate in normal mode (OC-192c). NOTE: The EPP has been designed to operate with the same Dataslice device as the PP, and this has certain limitations that the designer should be aware of. The following sections on subporting explain these limitations. Ports are numbered 0 through 31and each port has one EPP. Subports are denoted as 0 through 3. Ports with subports are shown as a (port, subport) pair; for example, port 4, subport 3 would be shown as (4,3) Ports without subports (OC-192c) are simply numbered. The following queueing description assumes an ETT1 core configured with all 32 ports in subport mode, supporting 128 OC-48c linecards. First, consider a single priority. Figure 12 shows the 128 counters needed by one OC-48c linecard. There are currently two requests outstanding for output (0,0). The ETT1 Chip Set uses virtual output queues to avoid head-of-line blocking issues, therefore, the EPP needs to keep track of unicast requests going to each one of the 128 possible output channels (0,0) through (31,3). Consequently, 128 counters are needed. These counters reflect the outstanding requests that this one OC-48c ingress linecard has made to the switch.
34
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 12. The Input Request Counters
EPP 0,0 2 0,1 request OC-48c linecard grant 0 0,2 3 0,3 7
31,3 0
The EPP can support up to four OC-48c channels, and each channel must have its own set of request counters for each of the 128 OC-48c output channels. Figure 13 shows the full set of counters for a single priority. Within the EPP an LCS Grant Manager process issues grants to the linecards in response to the received requests. Once per cell time the LCS Grant Manager selects one request counter to grant, given that many may qualify. A request counter qualifies to receive a grant if the counter is non-zero and if there is at least one empty cell buffer in the VOQ that is used for cells going to that particular output. Once the LCS Grant Manager has selected a particular VOQ, it decrements the corresponding counter and sends a grant to the appropriate linecard. The grant indicates a specific VOQ within the linecard. When the linecard receives the grant it does two things. First, it sends to the EPP the cell body that is currently at the head of the granted queue. Second, it increments a local credit counter that is used to determine whether a new request for that queue can be sent.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
35
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 13. Full Set of Counters for a Single Priority
EPP 0,0 2 0,1 request OC-48c linecard grant 0 0,2 OC-48c 0 3 0,3 7
31,3 0 0,0 5 request OC-48c linecard grant 31,3 2 0,0 0 request OC-48c linecard grant 31,3 7 0,0 15 0,1 request OC-48c linecard grant OC-48c 3 31,3 3 8 0,1 12 OC-48c 2 0,1 2 OC-48c 1
36
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.3.3.1
Virtual Output Queues
Ideally, the EPP/DS would have a separate VOQ which would correspond to each of the request counters. This would mean that congestion on one VOQ would not impact any other VOQ on the same OC-48c or any VOQs on different OC-48c channels. In practice, the EPP is constrained by the amount of cell buffering available in the ETT1 Dataslice device (which contains the actual cell buffers). Consequently, all four request counters that correspond to a single output channel share a single queue, as shown in Figure 14. The EPP only manages a single queue for each of the output channels (whether the output channel is OC-48c or OC-192c). If the EPP is connected to four OC-48c inputs, then those four OC-48c linecards must share the single queue used for each output. The four dotted arrows in Figure 14 show that the four request counters for output (0,0) all map to the same VOQ for output (0,0).
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
37
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 14. All Request Counters for a Single Output Share a Single Queue
EPP 0,0 2 0,1 0 OC-48c linecard OC-48c 0 0,2 3 0,3 7 0,1 0,2 0,3 0,0 5 0,1 OC-48c linecard OC-48c 1 1,1 31,0 2 0,1 0 0,2 OC-48c linecard OC-48c 2 31,0 31,1 0,0 15 0,1 OC-48c linecard OC-48c 3 31,3 3 8 31,3 31,2 12 1,2 1,3 2 1,0 0,0 (This is for cells going to output OC-48c channel 0 on port 0) Virtual Output Queues (in the Dataslices) (1 per output channel) EPP
31,3 0
31,3 7
38
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
The complete sequence of events at the input side is shown in Figure 15. 1. Linecard a issues a request to the EPP to send a cell to output channel (0,1). The initial request count is zero, so the arrival of the request then increments the request count to one. 2. The LCS Grant Manager sees that there is at least one empty cell buffer in the virtual output queue for output (0,1), and that the request counter for (0,1) is non-zero and so it decrements the request counter (back to zero) and issues a grant to the linecard. 3. The linecard then forwards the actual cell body to the EPP which stores the cell in the virtual output queue. The EPP also issues a request to the Scheduler, indicating that there is a cell waiting to be transferred to output (0). 4. Later the Scheduler issues a grant to the EPP which can forward the cell through the Crossbar to the output channel (the EPP at port 0).
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
39
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 15. Sequence of Events at the Input Side
EPP OC-48c 0 0,0 2 OC-48c linecard 0,1 0 0,2 3 0.3 7 0,3 0,2 Scheduler 0,1 Virtual Output Queues 0,0 Crossbar
Initial state. Subport 0 has no outstanding requests for output (0,1)
EPP OC-48c 0 0,0 Request (0,1) OC-48c linecard 2 0,1 1 0,2 3 0.3 7 0,3 0,2 Scheduler 0,1 Virtual Output Queues 0,0 Crossbar
Subport 0 issues a request to send a cell to output (0,1). The request count for output (0,1) is incremented. EPP OC-48c 0 0,0 2 OC-48c linecard Grant (0,1) 0,1 1 0,2 3 0.3 7 LCS Grant Manager 0,1 0,2 Scheduler 0,3 Virtual Output Queues 0,0 Crossbar
The LCS Grant Manager issues a grant for queue (0,1) to subport 0.
40
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 15. (Continued)
EPP OC-48c a 0,0 2 OC-48c linecard 0,1 0 0,2 3 0.3 7 0,3 0,2 Scheduler 0,1 Request (0) Virtual Output Queues 0.0 Crossbar
Linecard a receives the grant and forwards cell body to go to output (0,1). A request is made to the Scheduler.
EPP OC-48c a 0,0 2 OC-48c linecard 0,1 1 0,2 3 0.3 7 0,3 0,2 Scheduler 0,1 Grant (0 Virtual Output Queues 0,0 Crossbar
The Scheduler issues a grant to queue (0,1). The EPP then sends the cell through the crossbar to the output queue of port 0 .
The departure of a cell from the VOQ will mean that there is now space for at least one more cell in that VOQ, and that could trigger the LCS Grant Manager to issue a new grant, assuming that a new request had come in for that queue.
1.3.3.2
The Output Process
Each EPP consists of an input (i)EPP and an output (o)EPP. The request counters and virtual output queues described above all reside in the iEPP. The oEPP has virtual input queues that receive the cells forwarded via the Crossbar.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
41
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
In the ideal scenario, each oEPP would have a separate virtual input queue for every (possibly 128) input channel. However, the same restriction applies, limiting the oEPP to just 32 queues per output OC-48c (and per priority). Figure 16 shows the virtual input queues in each oEPP when attached to four OC-48c ports. Each virtual input queue maps back to a single virtual output queue in one of the iEPPs. In abstract terms, the virtual input queue is simply an extension of the appropriate virtual output queue. Each output OC-48c channel within the oEPP has its own set of unicast virtual input queues. If an output queue (virtual input queue) is backpressured, then this will push back to the appropriate virtual output queue. However, each virtual output queue is shared by the four input OC-48c channels within that iEPP, so backpressuring a virtual output queue has the effect of asserting backpressure to all four OC-48c's on the same iEPP. The oEPP has an Output Scheduler process which determines which cell should be forwarded to the egress linecards.
42
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 16. Virtual Input Queues
iEPP port 0 VOQ 0,0 Virtual Input Queues 0 VOQ 0,1 1 VOQ 0,2 output OC-48c 0 oEPP port 0
VOQ 0,3 31
VOQ 31,3 0 VOQ 0,0 1
iEPP port 1
output OC-48c 1
VOQ 0,1 31
VOQ 0,2
VOQ 0,3 0 VOQ 31,3 1
output OC-48c 2
31 iEPP port 31 VOQ 0,0
VOQ 0,1
0 1
VOQ 0,2
output OC-48c 3
VOQ 0,3 31 VOQ 31,3
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
43
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.3.3.3
Four Priorities
The description in the preceding sections only considered a single priority. The EPP supports four priorities. Therefore, there are four times as many counters and queues as have been described so far , and this is only for unicast traffic. The four priorities can be modeled as separate planes; the input and output queue figures shown earlier can be considered as 2-D planes. The four priorities are then four copies of these planes, stacked next to each other in a third dimension. Figure 17 illustrates this model for the reference counters and virtual output queues. The virtual input queues at the oEPP are not shown. Consequently, each iEPP has 2048 request counters (4 OC-48c input channels * 128 output channels * 4 priorities) and 512 virtual output queues (128 outputs * 4 priorities). Each oEPP has 512 virtual input queues (4 output channels * 32 ports * 4 priorities). Remember, this calculation is for the configuration of 128 ports of OC-48c. Later on we describe how the number of queues and request counters differs for some other possible configurations.
44
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 17. The Four Priorities
0,0 2 0,0 1b 2 0 0,0 1b 1c 2 0 3 0,0 OC-48c a 1b 1c 1d 2 0 3 7 OC-481c 1d 0,1 a 3 0 7 OC-48 0,2 1d a OC-48c 0 32d 3 7 0 0,3 32d 7 0 1a 32d 5 0 1a 1b 31,3 5 2 0 1a 1b OC-48c b 5 2 0,0 b OC-481b 32d 5 2 2 OC-48 0,1 b OC-48c 1 32d 2 2 32d 1a 2 0 1a 1b 31,3 2 0 12 1a 1b OC-48c c 0 12 1a c OC-481b 32d 0 12 7 1b OC-48 c OC-48c 2 32d 12 7 1a 32d 15 7 1a 1b 32d 15 8 7 1a 1b OC-48c d 15 8 1a d OC-481b 32d 15 8 3 OC-481b d OC-48c 3 32d 8 3 32d 3 32d 3 Priority 3 Priority 2 Priority 1 1a Priority 0 1a 1a 0,0 1b 0,1 1c 0,2 1d 0,3 2a 1,0 2b 1,1 2c 1,2 2d 1,3 32a 32a 32b 32a 32b 32c 31,0 32b 32c 32d 31,1 32c 32d 31,2 32d 31,3 2a 2b 2c 2d 1b 1c 1d 2a 2b 2c 2d 1b 1c 1d
1.3.3.4
Multiplexing Scheduler Requests and Grants.
Figure 16 and Figure 17 imply that the ETT1 Scheduler can accept requests from each of the 32 ports where a request specifies one of 128 output channels. This is not the case; the Scheduler only maintains information on a port basis, not a subport basis.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
45
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Consequently, the requests associated with cells going to subports (0,0), (0,1), (0,2) and (0,3) are amalgamated to become requests for port 0. NOTE: The channel information is not lost; a cell going to channel (0,1) will still be delivered to channel 1, port 0. However, the four queues have their requests multiplexed over a single request/grant connection. Figure 18 illustrates this concept.
Figure 18. Scheduler Requests/Grants are Multiplexed at a Port Level
VOQs in iEPP cells for output subport (0,0) cells for output subport (0,1) Request to send to port 0 cells for output subport (0,2) Grant to send to port 0 cells for output subport (0,3) Scheduler
The EPP has two decisions to make: which request to issue, and which of the four queues to grant. At any cell time, the iEPP may have several possible new requests which it must communicate to the Scheduler. The iEPP has a Scheduler Request Modulator which makes this decision. See Section 3.1.3 "Scheduler Request Modulator" on page 158 for a discussion of this algorithm. When the iEPP receives a grant from the Scheduler it must decide which of the four VOQs should be serviced. A simple round-robin algorithm is used (across non-empty queues).
1.3.4
Multicast Traffic (subport mode)
Multicast cells are handled at a port level, not a subport level. There is a single cell queue within the iEPP, for each priority of multicast cells. Each OC-48c channel has its own request counter, but, like the unicasts, these four request counters map to a single virtual output queue (per priority), as shown in Figure 19.
46
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 19. Each iEPP has a Single Virtual Output Queue for Multicast Cells
OC-48c 0
OC-48c 0
m 2 m 5 m 0 m 4
iEPP
Priority 0
OC-48c 1
OC-48c 1
multicast cells
OC-48c 2
OC-48c 2
OC-48c 3
OC-48c 3
As before, this single queue is not a limitation unless backpressure is asserted to the queue, in which case all four OC-48c subports are backpressured for all multicast cells from this port at this priority. ETT1 uses a multicast tag in the LCS header to identify the multicast group that this cell should go to. The iEPP then maps the tag into a port vector with one bit per port. This port vector is then passed to the Scheduler to indicate the list of ports that should receive this multicast cell. Again, the Scheduler only recognizes 32 ports and not 128 OC-48c subports; at the input side, a multicast cell destined for any one of output ports (0,0), (0,1), (0,2) or (0,3) would simply be forwarded to the oEPP at port 0. The oEPP receives the multicast cell, together with its tag. It then maps the (tag, source_port) into a new 4-bit vector that identifies what combination of the four subports should receive this multicast cell. Figure 20 shows the entire process.
Figure 20. Multicast Cell Processing
iEPP multicast Crossbar Output Multicast Input Multicast Fanout Table tag 17 12 4 src prt tag Fanout Table multicast oEPP
Scheduler 32
Output Scheduler
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
47
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
In order to reduce the blocking effects of one OC-48c channel over another channel on the same port, there is a special multicast output queue for each priority. The multicast cells forwarded through the Crossbar are still queued in the order they arrive. This special queue enables cells to be sent out-of-order, but will always be sent in-order for a given egress subport. Therefore, a multicast cell waiting in the output queue memory can only be blocked by cells destined to the same egress subport. Additional output queue memory becomes available whenever a cell is dequeued. If backpressure is asserted by one of the OC-48c channels to the output multicast queue, then subsequent cells going to other channels on the same port may be blocked if the entire queue is occupied by cells destined to the blocked OC-48c channel. This is a consequence of sharing the output queue memory allocation for multicast cells at a given priority. This can cause multicast backpressure to the central Scheduler which can block multicast cells destined to other ports at the same priority if multicast backpressure is enabled.
1.3.4.1
Multicast Tables (OC-192c and Quad OC-48c Modes)
The input multicast fanout table (EITIBM in the EPP) is always used for multicast cells. Every EPP has its own EITIBM table with 4096 entries. The EPP uses the 12-bit multicast tag field from the LCS label field to determine which entry in the table should be used for each multicast cell, as shown in Figure 21. Each entry is simply a 32 bit vector where a 1 means that the cell should go to that port. So if an entry has a 1 at bit 12, for example, then a multicast cell that uses that entry will be sent to port 12 (as well as any other ports that have their bit set to 1).
Figure 21. EITIBM structure
iEPP Input Multicast Fanout Table EITIBM 0 Multicast tag [11:0]
12 4095 To Scheduler 32
31
0
1 bit for each output OC-192c port
48
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
The output multicast fanout table (EOTIBM in the EPP) is only used if the port is in quad OC-48c mode. In quad OC-48c mode a multicast cell arrives at the oEPP via the crossbar. The oEPP must determine what combination of the four OC-48c ports that this multicast cell should be forwarded to (15 possible combinations assuming it goes to at least one of the ports). The oEPP has the multicast tag (12 bits) from the label, but that is not sufficient. It is not sufficient because the multicast tags are managed independently across all ports: each iEPP has its own EITIBM and so a given 12 bit multicast tag might be used to specify different multicast fanouts for two different iEPPs. To resolve the ambiguity, the oEPP must also include the source port number with the 12 bit multicast tag, and combines the 5 bit source port with the 12 bit multicast tag to produce a 17 bit tag that is unique across the entire fabric. This can then be used to index into a table of 4-bit entries as shown in Figure 22.
Figure 22. OITIBM structure
oEPP Output Multicast Fanout Table EOTIBM 0 Multicast tag [11:0] Source port[4:0]
17 131071 To Output Scheduler
4
3
0
1 bit for each of the four local OC-48c ports
The index is the combination of {Multicast_tag, source_port}, so that the source port forms the least significant bits of the address (index). Each entry has four bits such that bit 0 corresponds to subport 0 and bit 3 corresponds to subport 3.
1.3.5
Combinations of OC-192c and Quad OC-48c Linecards
A switch might contain both OC-192c ports and quad OC-48c ports. Each port card must be configured with information about all of the cards. The number of ingress queues is a function of this mix of ports: given n quad OC-48c ports and m OC-192c ports, then each port will have U unicast queues, where: U = 4 priorities * n * 4 egress subports + 4 priorities * m So if n = 32 and m = 0 then U = 512.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
49
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
If a port is operating in OC-192c mode then it will also have U unicast request counters. If a port is operating in quad OC-48c mode then it will have 4 * U unicast request counters - a separate set of request counters for each of the OC-48c linecards. Each port always has four multicast ingress queues. The number of egress queues is a function of the mode of the port itself, not the other ports: a port in OC-192c mode will have one queue for each input port and priority, or 4 * 32 = 128 egress queues. A port in quad OC-48c mode will have four times as many queues, or 512 queues. Each port always has four multicast egress queues. The size and depth of all request counters and queues are defined in Section 3.2 "Input Dataslice Queue Memory Allocation with EPP" on page 162 and Section 3.3 "Output Dataslice Queue Memory Allocation with EPP" on page 162.
1.3.6
Summary
Every ETT1 port has a number of ingress request counters and queues as well as egress queues. Given support for 32 ports, four subports per port, and four priorities, then there are up to 512 unicast ingress queues and four multicast ingress queues, and the same number of egress queues. At every cell time, the Scheduler arbitrates among all of the ingress queues that are eligible to forward a cell through the Crossbar. An ingress queue is eligible if it is non-empty and its destination egress queue is not backpressured. The Scheduler makes its decision using absolute priorities, with multicast requests having precedence over unicast requests at the same level, as shown in Figure 23.
Figure 23. Scheduler Priorities
Best Effort Traffic Priority 0 Multicast Priority 0 Unicast Priority 1 Multicast Priority 1 Unicast Priority 2 Multicast Priority 2 Unicast Priority 3 Multicast Priority 3 Unicast Lowest precedence Highest precedence
50
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.3.7
The LCS Protocol
In this section, we describe the LCS protocol from a functional perspective and explain how it works with the queueing model that was described in the previous section. NOTE: The LCS protocol is defined in the "LCS Protocol Specification -- Protocol Version 2", available from PMC-Sierra, Inc. This version of LCS supersedes LCS Version 1. Version 2 is first supported in the TT1 Chip Set with the Enhanced Port Processor device (also referred to as the ETT1 Chip Set) and will be supported in future PMC-Sierra products. A comparison of the two versions of the LCS protocol is in the same document. The primary role of the LCS protocol is as a credit-based flow control mechanism that ensures that cells are transferred between the linecard and the switch core only when sufficient buffer space is available. In order to do this, the linecard must have knowledge of the queues that exist in the ETT1 core. The LCS protocol is an asymmetrical protocol; we will first consider the ingress direction. Figure 24 shows the 132 best effort ingress queues in an ETT1 port (this assumes an all-OC-192c switch). The linecard may have a very different number of queues (typically many more), and thus must map its own internal queues to the physical queues present in the ETT1 core.
Figure 24. ETT1 Port Ingress Queues
Payload LCS Header
Linecard
ETT1 Port
Cell requests and payloads
Mapping 128 unicast linecard ingress queues 4 multicast 4 multicast 128 unicast Credit Information
For example, the linecard might provide an ATM interface which can manage many thousands of virtual circuit flows, and might have a separate internal queue for each flow. The linecard must map its own ingress queues to the 132 ingress queues present in the ETT1 port. The 132 queues shown in the linecard do not have to be physically implemented provided that the linecard can perform the requisite mapping from linecard queues to ETT1 queues on-the-fly. The ETT1 ingress queues are Virtual Output Queues; the mapping function simply has to map each linecard queue to the desired egress port within the ETT1 core. The mapping function must also select the appropriate priority if more than a single priority is required.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
51
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 24 shows ingress cell requests and data cells being forwarded to the ETT1 port. These cells are of a fixed size and format: an LCS cell consists of an eight byte LCS header followed by a fixed length payload. The payload can be 64 or 76 bytes (depending on the number of Dataslices and Crossbars used), but all ports in the switch must use the same size payload. The TT1 Chip Set never examines or modifies the cell payload unless the cell is used for internal control purposes. The linecard can send a new cell request whenever a new cell arrives at the linecard and the linecard has at least one request credit remaining for the appropriate queue. So the linecard must keep track of the number of outstanding requests that it can send, and must not issue a request unless it has a corresponding credit. The ETT1 port returns grant/credit information to the linecard. This grant/credit information reflects the occupancy of the ingress queues in the ETT1 port. The linecard can only send a cell to a given ingress queue within the ETT1 port when it receives a grant from the ETT1 port. The linecard increments its corresponding credit counter when it receives a grant/credit. In order to sustain the maximum possible throughput, the linecard must be able to send a new cell within a short time of receiving a grant/credit from the ETT1 port. If the linecard cannot do this then the VOQ at the ingress ETT1 port may become empty which might, in turn, affect the throughput of the switch. The system designer must be aware of the time budget that is really available to the linecard devices. The grant/credit mechanism is not used in the egress direction. Rather, a simpler Xon/Xoff-like mechanism is used. The oEPP will forward cells to the egress linecard whenever it has cells waiting in its output queues. The linecard can request the EPP to not send a cell of a given priority, or any combination of priorities, by asserting the hole request bits in the LCS header. If a hole request bit is asserted at a given priority, then it is a request from the linecard to the EPP that the EPP should not forward a cell of that priority at one cell time in the future. The time between the EPP receiving the hole request and observing it is guaranteed to be no more than 64 cell times. Future LCS compliant products will have different response times. Linecard designs requiring backpressure features should accommodate all LCS products to which they will attach. If a linecard has nearly exhausted its egress buffers for priority 2 cells, then it might continuously assert hole request for priority 2 until it can recover some additional buffers. NOTE: The hole request mechanism operates per-priority and not per-source port. Refer to the "LCS Protocol Specification - Protocol Version 2", available from PMC-Sierra, Inc., for further details on the LCS protocol and the definition of the various fields used within the LCS header in each direction.
1.4
EXPLICIT BACKPRESSURE FROM OEPP TO IEPP
The iEPP does not make a unicast request to the Scheduler unless the destination VIQ has a sufficient number of empty buffers. For each unicast egress VIQ there is one corresponding ingress VOQ. Every iEPP maintains state information on the occupancy of the relevant egress VIQs in all of the EPPs (including itself) in order to ensure that it does not forward a cell to a full VIQ.
52
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
The actual occupancy of a VIQ is dynamic: the number of free cell locations will increment whenever a cell is forwarded to the attached linecard, and the number of free cell locations will decrement whenever a new cell enters the VIQ from the crossbar. The iEPP knows the latter information - it is the iEPP that forwards the cell to the VIQ. But the iEPP does not know when a cell gets sent out to the linecard. So the oEPP must tell each iEPP how many cells have been sent from each relevant VIQ, and this information must be sent with sufficient frequency that the throughput of a single flow is not adversely affected. In the ETT1 fabric, this update information is transmitted from the oEPPs to the iEPPs via a second Crossbar, referred to as the Flow Control Crossbar. Figure 25 shows the cell flow from left to right and the reverse flow of the update information.
Figure 25. A Separate Crossbar is Used to Convey Backpressure Information to the iEPPs
Virtual Output Queues 0,0 0,1 0,2 0,3
sync
Virtual Input Queues Cell Crossbar 0,1 0,2 0,3 0,0
Req/Gnt
Scheduler
iEPP
oEPP
Flow Control Crossbar
Every iEPP maintains a set of counters (Output Queue Debit Counters) that track the number of cells in each of the relevant VIQs of every oEPP. At reset, these counters are all set to zero. A counter is incremented whenever the iEPP makes a request to forward a cell to the appropriate VIQ. If the counter reaches a pre-determined maximum, then the iEPP will not issue further requests. Once some cells are forwarded from the VIQ, the iEPP will reduce the counter and resume issuing requests for that flow. In general, the user does not need to be aware of the details of this mechanism except for the case when update information is lost, perhaps due to a noise-induced error on one or more of the links attached to the Flow Control Crossbar. The details of this error recovery mechanism are described in Section "Enhanced Port Processor - Flow Control Crossbar" on page 94.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
53
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.4.1
Flow Control Crossbar Synchronization
The Flow Control Crossbars interconnect every EPP with every other EPP. During each cell time, the Flow Control Crossbar sets the crosspoints such that every iEPP receives update information from one other oEPP. Further, no two iEPPs receive the same update information from the same oEPP. In the next cell time the crosspoints must be re-organized so that every iEPP receives update information from a different oEPP. The system must ensure that the Flow Control Crossbars and the EPPs are all synchronized, so that every iEPP knows from which oEPP it is receiving information at any given time. This synchronization information is generated by the Scheduler using the FCCSYN register to send a periodic pulse out to all EPPs. Further, if a redundant Scheduler is used then the two Schedulers must be synchronized. This Scheduler-to-Scheduler synchronization occurs after a Scheduler Refresh operation. Refer to Section 1.9.4 "Scheduler Refresh Procedure" on page 98.
1.5
TDM SERVICE
The TDM service enables linecards to periodically reserve bandwidth within the ETT1 core. Cells that are switched via this service will have a fixed latency through the switch and will not be delayed due to best effort contention for an output port. 1-in-N idle cells and egress control packets may cause temporary delays, but these should be absorbed by the guardband and speedup. The TDM service is implemented within the ETT1 core using: * * * dedicated queues in the ETT1 ports a table-based reservation mechanism in the ETT1 ports a flexible synchronization mechanism
1.5.1
TDM Queues
The linecard uses the normal LCS request-grant mechanism for TDM cells. When TDM cells arrive at an ETT1 ingress port, they are queued in a single TDM queue which can buffer up to 96 cells. The cells are stored in the ingress queue until their reservation time occurs, at which time they are forwarded through the crossbar to the appropriate egress TDM queue. Given that there can be no contention for TDM cells then a TDM queue might seem unnecessary. The queue is present so that the linecard does not need to be closely synchronized with the ETT1 fabric. If the port is operating in subport mode then four ingress queues are used, one for each subport. Each ETT1 port has a single egress TDM queue, even when the port is in subport mode. The ETT1 can transmit cells out-of-order between the subports, and so a single shared TDM queue does not cause a problem. Figure 26 illustrates the queueing structure.
54
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 26. The Queueing Structure
ETT1 Core
Linecard
ETT1 port
ETT1 port
Linecard
LCS credits
Ingress TDM queues
Figure 26 also highlights some important points with respect to the TDM service: * The linecard may also require a TDM ingress and egress queue. Unless the arrival of TDM cells at the linecard is synchronized with the start of the TDM Frame, the linecard will need to buffer those cells until the appropriate time. There is no backpressure from the ETT1 egress TDM queue. Backpressure is not meaningful for a TDM service within the ETT1 core. The egress linecard must accept TDM cells that are sent to it (or else discard them). The normal LCS credit mechanism is used for ingress TDM cells.
*
*
1.5.2
TDM Reservation Tables
The TDM service operates by enabling ports to reserve bandwidth through the ETT1 crossbar at specific times. Each port has two logically separate tables; one for sending and one for receiving TDM cells. The Send and Receive tables are physically implemented in a single table called the TDM table. This TDM table has 1,024 entries, although fewer entries can be used if the Frame length is less than 1024. Each entry is of the form shown in Table 2.
Table 2. TDM Reservation Table
Field Input VLD Input Tag
Size (bits) 1 10
ETT1 Crossbar
Ingress TDM queue
Egress TDM queue
Egress TDM queue
Meaning 1 if the input side TDM entry is valid Tag for incoming cells (TDM flow ID)
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
55
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 2.
TDM Reservation Table (Continued)
Field Subport Output VLD Input Port Output vector
Size (bits) 2 1 5 4
Meaning Identifies the source subport (if in subport mode, else 0) 1 if the output side TDM entry is valid Input port that the output expects to receive from A four-bit vector which defines the fanout of the subports that should receive a TDM cell.
The EPP contains two TDM tables so that one table can be used by the fabric while the system software is configuring the other table. All EPPs must use the same table at the same time. A TDM LCS Control Packet is used to specify which of the two tables is being used at any given time. At the start of the Frame, a TDM pointer in each port begins at entry 0 in the table and advances one entry every cell time. During every cell time, if the "current" TDM table Send entry is valid, the EPP checks to see if the cell at the head of the appropriate TDM queue has a TDM flow ID (tag) that matches the entry in the table. If not, then nothing special is done. If they do match then the EPP informs the Scheduler that it will be sending a TDM cell at some fixed time in the future. This prevents the Scheduler from trying to schedule a best-effort cell from that port at the same time. Figure 27 shows the logic at the ingress port.
56
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 27. Ingress Port Logic
TDM Cell Queues
OC-48c 0
OC-48c 1
OC-48c 2
OC-48c 3
=?
"Send"Table OC-48c Select TDM Identifier 0 991
Current TDM Slot Time
The egress side of the port also looks at the TDM table. If the Output Valid bit in the current entry is set, then the port tells the Scheduler that it expects to receive a TDM cell at some fixed time in the future. This prevents the Scheduler from trying to schedule a best-effort cell to that port at the same time. If the Scheduler doesn't receive the input TDM request bit from the specified input port, then it assumes that there is no TDM cell ready to be transmitted, and thus any egress port that expected to receive that cell will now become available for best-effort cell scheduling. When the TDM cell enters the egress port, it is placed in a separate, 96-entry, egress queue dedicated for TDM traffic. Whenever this queue is non-empty, the cell at the head of the queue is sent immediately to the linecard (provided that the Output Scheduler is not frozen).
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
57
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
The TDM service is inherently a multicast service. The egress ETT1 must therefore support the ability for a TDM cell to go to any combination of the four subports within a port. A four-bit field in every TDM table entry provides the multicast fanout for each TDM cell that is received at the egress port. The EPP keeps track of this fanout information and sends the TDM cell to each subport that is a member of the fanout. Figure 28 illustrates how the EPP uses the TDM table to determine the source port of a TDM cell, as well as the local subport multicast fanout.
Figure 28. The oEPP Must Treat the Transferred TDM Cell as a Multicast Cell
Cell from Crossbar Output TDM Cell Queue
0101
0111
0001
To Scheduler
Input port = 12
output vector = 0101 (multicast fanout) TDM Reservation Table
0
991
Current TDM Slot Time
The output TDM cell queue does not need to be large as TDM cells are always forwarded to the output linecards at the highest priority, and the TDM queue is never backpressured. The size of this queue does, however, restrict the possible TDM table configuration. The following constraints are required to prevent dropped TDM cells: * In a TDM table time slot, an input port can not send more than one cell and an output port can not receive more than one cell. Subport mode: The TDM Reservation Table can specify the same ingress subport only once every four time slots. Subport mode: The TDM Reservation Table can not specify the same egress subport more than 48 time slots within a moving window of 192 time slots. This constraint is caused by the TDM output queue.
*
*
58
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.5.3
TDM Frame Timing
A single TDM table is shown in Figure 29. In addition to the 1,024 entry portion, there is an additional "guard band" of G entries preceding the table. These G entries are unusable and are explained below. An additional 16 time slots (plus uncertainty/jitter in TDM sync timing) are unusable following the last valid entry in any of the TDM tables due to pipeline latency through the Scheduler and EPP. These 16 entries cannot be used for TDM traffic.
Figure 29. Single TDM Table
TDM_Sync Guard Band TDM_Period Main TDM Frame Unusable 16 entries + uncertainty TDM_Sync
G Entries (128 cells)
N Entries 0 N 1024
The time between TDM Sync signals is referred to as the TDM period. This period is set to the sum of G plus the length of table N plus the final latency allowance of 16 + uncertainty. In the EPP chapter, G is referred to as the "TDM offset value".
1.5.4
TDM Synchronization
There are several different levels of synchronization required for the TDM service to work properly. First, all EPPs must point to the same entry in their TDM tables in order for the Send and Receive reservations to match up in the Scheduler. Second, each linecard must be synchronized (approximately) with its ETT1 port so that it can send TDM cells at the right time and third, so that the linecard will switch TDM tables synchronously with the ETT1 port. The first level of synchronization, between ETT1 ports, is handled automatically by the ETT1 Chip Set. All of the ETT1 devices operate cell synchronously. The second and third levels of synchronization, between the linecard and ETT1, are handled by the TDM Sync mechanism. The Scheduler is configured in one of three ways, described below, to periodically send out a TDM_Sync to all ports. All of the ETT1 ports receive the TDM_Sync signal from the Scheduler at the same cell time. Upon receiving the TDM_Sync from the Scheduler, every port then (1) flushes any cells currently in the TDM input queue, (2) transmits a TDM Control Packet to the linecard and (3) starts a counter which increments every cell time. When this counter reaches the value of the TDM offset (guardband), the EPP begins comparing the tag of the cell at the head of the TDM FIFO with the entry in TDM table location 0, and normal TDM processing commences. The TDM_Sync and TDM Control Packet both include a Frame Select bit to tell the ports and linecards which TDM table to use. This allows all ports and linecards to switch TDM tables synchronously.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
59
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
TDM_Sync signals do not arrive at the linecards at exactly the same cell time. There may be several cell times between the first linecard receiving TDM_Sync and the last. The variation is due to variations in link lengths and also due to timing uncertainties when signals cross asynchronous clock domains. The guard band absorbs these variations, so the linecard does not need to be aware of the link delays or clock domain issues. As soon as the linecard receives the TDM_Sync signal it starts to forward TDM requests to the ETT1 port. The port has an ingress queue for these cells and so can buffer them until they are required. The ingress queue is 96 cells deep, and is controlled by the normal LCS request-grant flow-control mechanism. The linecard must forward TDM cells in the correct order (corresponding to its entries in its TDM Send table), but it does not need to be closely synchronized to the ETT1 core. The linecard can skip TDM cells (i.e. not send them), but must not re-order cells relative to the TDM tables. The ETT1 core waits for 128 cell times (the TDM offset) after it has sent TDM_Sync before the new TDM Frame is started. This provides all linecards with sufficient time to receive the TDM_Sync signal and to request and send in the first TDM cells. Figure 30 shows the sequence of events in more detail.
Figure 30. TDM Synchronization
Last usable slot in previous Frame
TDM_Sync arrives at all EPPs
TDM cells arrive at ETT1 ports from linecards
16 slots +uncertainty unusable
TDM_Sync arrives at the linecards at different times.
New Frame starts
The Scheduler can be configured to send the TDM_Sync in three ways. Each way is driven by one of these three sources: 1. A Suggested_TDM_Sync from a counter in the EPP. 2. A counter in the Scheduler. 3. A Suggested_TDM_Sync from a designated linecard (through the EPP). There are pros and cons to each of these methods. Each method is described in more detail below.
60
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.5.4.1
TDM Sync Driven by the EPP
The EPP can be programmed to send a Suggested_TDM_Sync to the Scheduler once every N+1 celltimes by writing N to the EPP's TDM Sync Generation Control register and writing 1 to the Enable TDM Sync Generation register. The TDM table select is also a field of the TDM Sync Generation Control register. The Scheduler is programmed to listen to the Suggested_TDM_Sync from a designated port. The advantage of this method over the third method is that the only exposure to jitter (uncertainties in delay) on the TDM_Sync is after the EPP sends the TDM Control Packet to the linecard, due to crossing asynchronous clock domains (and subport-ordering delay in subport mode). The advantage of this method over the second method is that when using two Schedulers for fault tolerance, the second method requires synchronous OOB writes to program the Schedulers at exactly the same time; this method provides a synchronous Suggested_TDM_Sync to the two Schedulers instead. A possible disadvantage of this method is that TDM Sync is synchronous to the ETT1's core clock, not to the linecard's clock or any other external synchronization signal.
1.5.4.2
TDM Sync Driven by the Scheduler
The Scheduler can be programmed to generate its own TDM_Sync signals once every N+1 celltimes using the Scheduler's TDM Control register. The TDM table select is also controlled via the TDM Control register. The Scheduler ignores Suggested_TDM_Sync from all ports. The advantages of this method are simplicity and low jitter, similar to the first method. The disadvantage of this method is that when using two Schedulers for fault tolerance, this method requires synchronous OOB writes to program the Schedulers at exactly the same time; if the Schedulers' TDM Control registers are enabled to send TDM Sync at different times, then all EPPs will report mismatching frames from the two Schedulers. Scheduler refresh will not synchronize Scheduler-generated TDM Syncs.
1.5.4.3
TDM Sync Driven by a Designated Linecard
Any linecard can send a Suggested_TDM_Sync to its ETT1 port, using a LCS TDM Control Packet, every TDM_Period cells. The TDM Control Packet also specifies which of the two TDM tables should be used. When the EPP receives a TDM Control Packet it sends a Suggested_TDM_Sync signal to the Scheduler. The Scheduler is programmed to listen to the Suggested_TDM_Sync from a designated port. The advantage of this method is that an external synchronization source can be used. The disadvantage of this method is that there is much more exposure to jitter in TDM_Sync. Asynchronous clock domain crossings (and subport-ordering delays in subport mode) will cause delay variations at multiple stages: the LCS request for the Suggested_TDM_Sync Control Packet, the LCS grant, the delivery of the Suggested_TDM_Sync Control Packet payload, and finally the delivery of the TDM_Sync Control Packet from the ETT1 to the linecards. Additionally, the Scheduler's internal digital TDM_Sync PLL
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
61
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
(described below) adds jitter. See the "TDM Service in the ETT1 and TTX Switch for a more thorough discussion of TDM_Sync jitter and its consequences.
Cores" App Note
On receiving a Suggested_TDM_Sync signal, the Scheduler computes the period at which the Suggested_TDM_Sync signals are arriving. The Scheduler then uses this period to determine the next period at which it sends a TDM_Sync to all ETT1 ports. The period of the TDM_Sync signals will track the period of the incoming Suggested_TDM_Sync signals. However the phase of the TDM_Sync signals may be shifted relative to the Suggested_TDM_Sync signals. Internally the Scheduler has a digital PLL which will gradually cause the transmitted TDM_Sync signal to have a fixed phase offset relative to the incoming Suggested_TDM_Sync signal. Figure 31 shows the passage of signals.
Figure 31. Signal Flow
ETT1 core Linecard TDM Control Packet Suggested_TDM_Sync Designated Linecard TDM Control Packet TDM Control Packet ETT1 Port TDM_Sync ETT1 Scheduler ETT1 Port TDM_Sync
Linecard TDM Control Packet
ETT1 Port
TDM_Sync
After a count of TDM_Period cell times, the designated line card will send another Suggested_TDM_Sync, the line card TDM period counter will re-initialize and the process will repeat. If the Scheduler sees three TDM periods go by without receiving another Suggested TDM sync, it will trigger an interrupt and continue to send TDM_Syncs to the ports at the current period.
1.5.5
Configuring the TDM Service
Before TDM cells can flow the system must be configured. This involves the following: 1. Configure the TDM tables in all ports that will send or receive TDM cells. 2. Configure the Scheduler in the appropriate mode for TDM synchronization. 3. Enable TDM cells.
62
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Configuring the tables is straightforward, as described in Section 1.5.2 "TDM Reservation Tables". The rules for the tables are: (a) No port can source more than one TDM cell at any cell time. (b) No port can receive more than one cell at any cell time. (c) The linecard must send the cells to the EPP: - in order, and - before the current TDM slot reaches the slot reserved for that cell. (d) Subport mode: The TDM Reservation Table can specify the same ingress subport only once every four time slots. (e) Subport mode: The TDM Reservation Table can not specify the same egress subport more than 48 time slots within a moving window of 192 time slots. This constraint is caused by the TDM output queue. Item (c) above is important to understand: the EPP will buffer all ingress TDM cells before they are sent through the ETT1 fabric. The EPP can only buffer 96 TDM cells at the ingress. The linecard must be able to send TDM cells to meet the average and the burst rate that are implied by the TDM table configuration. So, if the linecard is operating at 20M cells/sec, for example, while the ETT1 fabric operates at 25M cells/sec then the linecard must not reserve a contiguous burst of 128 TDM slots because it would be unable to supply the last few cells in time for their reservations. The next step is to configure the Scheduler. The Scheduler TDM service is controlled via its TDM Control register. The first aspect to configure is the source of the Suggested_TDM_sync signal. If a linecard or EPP will supply the Suggested_TDM_sync signal then configure the TDM Control register with the selected port and set the Current Port valid bit. If a linecard or EPP does not supply the Suggested_TDM_ sync signal then the Scheduler must be configured with the desired TDM_Period: write the desired period into the Initial Period field and toggle (0 to 1 to 0) the Load Initial Period control bit. In all cases the Enable bit (bit 31) of the Scheduler's TDM Control register should be set to 1. The final stage of configuring the Scheduler is to write the desired value to the Enable TDM traffic register. Each of the 32 bits should be set to 1 for those ports that will send or receive TDM traffic.
1.5.6
Changing the Source of the Suggested Sync
If the Scheduler is configured to use an external (port or linecard) source for the Suggested_TDM_sync, then you can change the source port by writing a new value into the port field of the TDM Control register. Once the new port starts to supply the Suggested_TDM_sync signal then the Scheduler will adjust the frequency and phase of the transmitted TDM_Sync signal to match that of the new incoming signal. The Scheduler will change the frequency of the outgoing sync signal once it has seen three Suggested_TDM_ sync signals, but it will only gradually adjust the phase. The phase of the outgoing TDM_Sync will be adjusted by one cell time in every TDM_period. NOTE: The phase reference for the Scheduler is the received Suggested_TDM_sync signal,
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
63
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
which will be several cell times after the linecard has sent the Suggested_TDM_sync to the ETT1 port. If the Scheduler does not see an incoming Suggested_TDM_sync signal for three TDM_periods then it will issue an interrupt indicating that the sync source has failed. However it will continue to operate with the current values of frequency and phase.
1.6
ETT1 USAGE OF THE LCS PROTOCOL
This section provides details of how the LCS header is used in an ETT1 system. Further information on the operation of the LCS protocol is in the "LCS Protocol Specification -- Protocol Version 2", available from PMC-Sierra, Inc.
1.6.1
LCS Protocol Prepend
LCS segments are 72 or 84 bytes in length. The line to switch format is shown in Table 3, while Table 5 shows the field definitions of this format. Likewise, Tables 3 and 4 show the switch to line format and field definitions, respectively.
64
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 3.
Bit 7 Byte 0 Req Valid Request Field
LCS Format from Linecard to Switch
Bit 6 Bit 5 Bit 4 Bit 3 Label_1 [13:7] Bit 2 Bit 1 Bit 0
Byte 1
Label_1 [6:0]
Type_1 [3] Rsvd
Byte 2 Payload Valid Payload Field
Type_1 [2:0] Seq_Number[9:3]
Byte 3
Byte 4 Byte 5 Byte 6
Seq_Number[2:0] Payload_Hole_Request[3:0] CRC-16[15:8]
Rsvd Rsvd
Hole Field
CRC Field Byte 7 Bytes 8-71 or 8-83 CRC-16 [7:0] Payload Data
Payload
Table 4.
Field Req Valid Label_1 Type_1 Payload Valid Seq_Number Payload_Hole_ Request Size (bits) 1 14 4 1 10 4
Definitions of LCS Fields from Linecard to Switch
Definition Indicates that request field is valid (Request Label_1). Label that is being requested (e.g. unicast-QID/multicast-label/TDM-label). Segment type that is being requested (e.g. Control Packet). Indicates that the Payload Data field contains a valid payload. A sequence number for synchronizing grants with payloads. Request for a "hole" in the payload stream from switch to linecard. (Priority 3,2,1,0 for bits [3:0], respectively.) (See Section 1.2.4 "End-to-End Flow Control" on page 22 for explanation). 16-bit CRC calculated over complete 64-bit prepend.a Payload Data
CRC-16 Payload Data
16 64 or 76 bytes
a. See Section 1.6.4.1 "Use of CRC-16 Field" on page 73 for details of CRC calculation and Section 3.4.2.2 "Reset and Control" on page 169, bit 4.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
65
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 5.
Bit 7 Byte 0 Grnt Valid
LCS Format from Switch to Linecard
Bit 6 Bit 5 Bit 4 Bit 3 Label_1 [13:7] Bit 2 Bit 1 Bit 0
Byte 1 Grant Field Byte 2 Type_1 [2:0]
Label_1 [6:0]
Type_1 [3] Seq_Num [9:5] Payload Valid Label_2 [13:12]
Byte 3
Seq_Num [4:0]
Byte 4 Payload Field Byte 5 Byte 6 CRC Field Byte 7 Bytes 8-71 or 8-83 Label_2 [3:0]
Label_2 [11:4] Type_2 [3:0] CRC-16 [15:8] CRC-16 [7:0] Payload Data
Payload
Table 6.
Field Grnt Valid Label_1 Type_1 Seq_Num Payload Valid Label_2 Type_2 CRC-16 Payload Data Size (bits) 1 14 4 10 1 14 4 16 64 or 76 bytes
Definitions of LCS Fields from Switch to Linecard
Definition Indicates that grant field is valid (Grant Label_1 and Sequence Number). Label for credit (e.g. QID/mcast-label/TDM-label). Segment type for credit (e.g. TDM). Sequence Number associated with grant. Indicates whether the egress payload is valid (Payload Label_2 and Payload Data). Label for egress payload (e.g. ingress port identifier). Segment type for egress payload (e.g. Control Packet). 16-bit CRC calculated over complete 64-bit prepend.a Payload Data
a. See Section 1.6.4.1 "Use of CRC-16 Field" on page 73 for details of CRC calculation and Section 3.4.2.2 "Reset and Control" on page 169, bit 4.
66
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.6.2
Use of Label and Type Fields
This section provides detail about the generic 14-bit labels and 4-bit types defined in Table 7, Table 8, Table 9, and Table 10. In all cases, in multi-bit fields, the MSB is transmitted first. For each segment, the "Type Field" and "Label Field" are encoded as follows:
Table 7. Encoding of 4-bit Type Fields
Type[3:0] 0000 Segment Type Control Packet Use Provides for inband control information between the linecard and the switch core. Currently not defined for use. TDM service provides preallocated bandwidth at a regular time interval. Prioritized unicast service where priority 0 is the highest priority. Prioritized multicast service where priority 0 is the highest priority.
0001,...,0110 0111
Reserved for future use TDM
1000,...,1011
Unicast Best-Effort (Priority 0,1,2,3)
1100,...,1111
Multicast Best-Effort (Priority 0,1,2,3)
Table 8.
Segment Type Control Packet Type_1 0000
Usage of the Ingress Request Label_1
Encoding of Request Label_1[13:0] Rsvda (13), CP-category (1). CP-category = 0 for LCS Control Packets; CP-category = 1 for CPU Control Packets. MUX (2)b, Expansionc (2), TDM Tag (10) MUX (2)b, Expansiond (5), Destination Port (5), Destination subporte (2) MUX (2)b, Multicast Tag (12)
TDM Unicast Best-Effort (Priority 0,1,2,3) Multicast Best-Effort (Priority 0,1,2,3)
0111 10xx
11xx
a. Reserved. All bits labeled Rsvd must be set to zero. b. The MUX[1:0] field is placed at the beginning of the Request Label_1 field in all segments (i.e. both data traffic and control packets). If this field is not used for a MUX function, it can be used as additional bits for expansion. If not used, it must be set to zero. c. TDM Expansion field used in products if the number of TDM slots is increased. If this field is used, the "TDM Tag" field will increase contiguously. If not used, must be set to zero. d. Unicast Expansion field used in products if the number of ports is increased. If this field is used, the "Destination Port" field will increase contiguously. If not used, must be set to zero. e. If the port has no subports (i.e. if it is connected to a single linecard without multiplexing), this field must be set to zero.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
67
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 9.
Segment Type Control Packet Type_1 0000
Usage of the Egress Grant Label_1
Encoding of Grant Label_1[13:0] Rsvda (13), CP-category (1). CP-category = 0 for LCS Control Packets; CP-category = 1 for CPU Control Packets. MUX (2)b, Expansionc (2), TDM Tag (10) MUX (2)b, Expansiond (5), Destination Port (5), Destination subporte (2) MUX (2)b, Multicast Tag (12)
TDM Unicast Best-Effort (Priority 0,1,2,3) Multicast Best-Effort (Priority 0,1,2,3)
0111 10xx
11xx
a. Reserved. All bits labeled Rsvd must be set to zero. b. The MUX[1:0] field is placed at the beginning of the Grant Label_1 field in all segments (i.e. both data traffic and control packets). If this field is not used for a MUX function, it can be used as additional bits for expansion. If not used, it must be set to zero. c. TDM Expansion field used in products if the number of TDM slots is increased. If this field is used, the "TDM Tag" field will increase contiguously. If not used, must be set to zero. d. Unicast Expansion field used in products if the number of ports is increased. If this field is used, the "Destination Port" field will increase contiguously. If not used, must be set to zero. e. If the port has no subports (i.e. if it is connected to a single linecard without multiplexing), this field must be set to zero.
Table 10.
Segment Type Control Packet Type_2 0000
Usage of the Egress Payload Label_2
Encoding of Payload Label_2[13:0] Rsvda (13), CP-category (1). CP-category = 0 for LCS Control Packets; CP-category = 1 for CPU Control Packets. Expansionb (7), Source Port (5), Source subportc (2) Expansiond (7), Source Port (5), Source subportc (2) Expansione (7), Source Port (5), Source subportc (2)
TDM Unicast Best-Effort (Priority 0,1,2,3) Multicast Best-Effort (Priority 0,1,2,3)
0111 10xx
11xx
a. Reserved. All bits labeled Rsvd must be set to zero. b. TDM Expansion field used in products if the number of ports is increased. If this field is used, the "Source Port" field will increase contiguously. If not used, must be set to zero. c. If the source port has no subports (i.e. if it is connected to a single linecard without multiplexing), this field will be zero. d. Unicast Expansion field used in products if the number of ports is increased. If this field is used, the "Source Port" field will increase contiguously. If not used, must be set to zero. e. Multicast Expansion field used in products if the number of ports is increased. If this field is used, the "Source Port" field will increase contiguously. If not used, must be set to zero.
68
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.6.2.1
Subport Mode Label and MUX Fields
Label fields are arranged such that port addresses can be subdivided into four multiplexed, subport addresses. Each segment type with a subport field utilizes bits [1:0] for the subport addresses. When subports are not utilized, these bits will be set to zero. Subport multiplexing shares a common physical connection to carry segments to/from each subported linecard to a switch port. The time multiplexing between subports may be done on a quad linecard, or may be done at an intermediate stage. When time-multiplexed, segment transmissions are sequenced between each of the four subports through the use of a MUX field. When a port of an LCS device uses the time-multiplexed subport mode, two bits in the Request Label_1 field of ingress and Grant Label_1 field of egress segments are used to multiplex/demultiplex each of four subport channels. The 2-bit field is called the MUX[1:0] field. For ingress segments, the MUX[1:0] bits are carried in the Request Label_1 field; for egress segments, the MUX[1:0] bits are carried in the Grant Label_1 (but not the Payload Label_2). Note that for ingress traffic, this field is used to designate the source subport of the ingress segment and is independent of the Request Label_1 subport field. Likewise for egress traffic, the MUX field is used to designate the destination subport of the egress segment and is independent of the Grant Label_1 and Payload Label_2 subport fields. Segments entering the switch core must have the MUX[1:0] field correctly set. The MUX[1:0] field in arriving segments must follow the sequence 00,01,10,11,00,... in consecutive segments. The MUX[1:0] field in departing segments must also follow the sequence 00,01,10,11,00,... in consecutive segments. When the MUX subport sequence must be interrupted by the need to send an Idle frame, the sequence should continue with the next subport in sequence following the previously sent subport. The MUX[1:0] field is placed at the beginning of the Request Label_1 or Grant Label_1 field in all segments (i.e. both data traffic and control packets). Segments departing or arriving at the subported line card may have the MUX[1:0] field set to correspond to the linecard's designated subport. Optionally, the linecard may set the MUX[1:0] field to zero, relying on an intermediary multiplexer device to correctly set the MUX[1:0] field. In such a configuration, the LCS prepend CRC is generated and checked masking the MUX[1:0] field to zero, enabling the intermediary multiplexer device to independently set the MUX[1:0] field without termination and regeneration of the CRC field. Note that without the requirement of setting the MUX[1:0] field, the linecard's framing function does not have to be subport aware in a system with an intermediary, time-multiplexed data stream.
1.6.3
Control Packets
The LCS link is controlled by Control Packets (CPs) that are generated by the linecard or the switch core. When the switch core sends a CP to the linecard, it marks the correct Payload Label_2 and Type_2 fields as a Control Packet and sends the control information in the payload data. When the linecard wishes to send a CP to the switch core, it goes through the normal three phase request/grant/transmit process. Control Packets have a higher priority over other traffic; therefore, grants will be returned in response to CP request prior to other outstanding requests. There are three categories of CPs: (1) LCS Control Packets that are used to perform low-level functions such as link-synchronization, (2) CPU Control Packets which are used for communication between the
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
69
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
linecard and the switch core CPU, and (3) Single Phase Control Packets which are used as a special form of LCS Control packets for timing sensitive information.
1.6.3.1
LCS Control Packets
LCS CPs are specified by setting the Label_1/Label_2 Field to 14'h0000 (CP-category for LCS Control Packets) and associated Type Field to 4'h0 (Type for Control Packets). Table 11 describes the contents of the first 10-bytes of the LCS segment payload for an LCS CP format.
Table 11.
Field CP_Type CP_Type Extension Size (bits) 8 56
LCS Control Packet Format
Definition Indicates the type of LCS Control Packet (e.g. TDM)
Indicates the information carried with this type of control packet. Unused bits must be set to zero. 16-bit CRC calculated over 10-byte control packet payload.a
CRC-16
16
a. See Section 1.6.4.1 "Use of CRC-16 Field" on page 73 for details of CRC calculation.
Because there is no flow control mechanism for LCS CPs sent to the switch fabric, and a CP request/grant does not distinguish the CP_Type, care must be taken so as not to overflow the receiving CP queues. At most, only one request for each CP_Type should be outstanding. In what follows, the three CP_Types of currently defined LCS CPs are described. NOTE: The actual LCS headers for Switch-to-Linecard Control Packets are not built in to the EPP. Instead, the EPP expects to find the headers, preconfigured at specific queue locations within the Dataslices. Thus, part of the initialization sequence of a new ETT1 port card is to write the values of the Control Packets into the Dataslice queues. Section 3.3 "Output Dataslice Queue Memory Allocation with EPP" on page 162, provides details of these locations.
70
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
TDM Control Packets The format of TDM CPs is described in Table 12.
Table 12.
Field CP_Type TDM Frame Sel Unused CRC-16 Size (bits) 8 1 55 16
TDM Control Packet Format
Definition 00000000 Indicates which TDM frame to use. Set to zero. 16-bit CRC calculated over 10-byte control packet payload.
Note that when a TDM CP is sent from the linecard to the switch core, no reply is generated by the switch core. Request Count Control Packets Request Count CPs are only sent from the linecard to the switch, and are used to keep request counter values consistent between the linecard and the switch core. When the switch core receives this CP, it updates the corresponding request counter (specified by Label_1 and Type_1) with the count field. This control packet can be sent periodically or when the linecard suspects that the linecard and switch core have different values (e.g. following a CRC error). Request Count CPs are required for unicast traffic and also required for multicast and TDM requested traffic when supported. For unicast traffic, the Label_1 field is set as described in Table 8. For multicast traffic, the Label_1 field is undefined and should be set to zero. The format of the Request Count CP is described in Table 13
Table 13. Request Count Control Packet Format
Field CP_Type Label_1 Type_1 Count Unused CRC-16 Size (bits) 8 14 4 14 24 16 Definition 00000001 Label for request counter. 10xx unicast 11xx multicast Update counter value, always set to zero for multicast. Set to zero. 16-bit CRC calculated over 10-byte control packet payload.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
71
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Start/Stop Control Packets Start/Stop Control Packets are used to toggle the flow of data traffic over the link (here, data traffic is defined to be TDM, best-effort unicast and multicast segment payloads and associated requests and grants. i.e. all non-Control Packet segments). Start/Stop Control Packets can be sent over an LCS link in either direction. If a linecard sends a Stop Control Packet to the switch core, the switch core will stop sending grants to the linecard for ingress data traffic, and will stop sending egress data traffic to the linecard. When a Start Control Packet is sent, grants and egress data traffic resume. If the switch core sends a Stop Control Packet to a linecard, the linecard must stop sending new data traffic requests into the switch core until a Start Control Packet is sent. Note that when either end has received a Stop Control Packet, it can continue to request, grant, send and receive all categories of Control Packets. When an LCS link powers-up, it will be in Stop mode: i.e. it will act as if a Stop Control Packet has been sent in both directions.
Table 14.
Field CP_Type Start/Stop Unused CRC-16 Size (bits) 8 1 55 16
Start/Stop Control Packet Format
Definition 00000010 1=Start, 0=Stop. Set to zero. 16-bit CRC calculated over 10-byte control packet payload.
1.6.3.2
CPU Control Packets
CPU Control Packets are specified by setting the Label_1/Label_2 Field to 14'h0001 (CP-category for CPU Control Packets) and associated Type Field to 4'h0 (Type for Control Packets). When the linecard sends a CPU CP to the switch core CPU, the complete payload data is passed to the switch core CPU. Likewise, when the switch core sends a CPU CP to the linecard, the complete payload data is passed to the linecard. No flow control mechanism is explicitly implemented for CPU CPs, and will be implementation dependent.
72
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.6.4
1.6.4.1
Use of CRC Fields
Use of CRC-16 Field
Control Packets and LCS protocol segment prepends use the CRC-16 polynomial: This is the same polynomial used in X.25. 1. Prior to starting the calculation, the remainder is set to 0xffff.
x 16 + x 12 + x 5 + 1 .
2. In all cases, the CRC-16 is calculated over all bits (80-bits of the CP or 64-bits of the prepend) with the CRC field set to all zeroes. 3. Transmission of the CRC-16 is done with the most significant bit sent first. Note that the use of the CRC-16 option in the ETT1 Switch to Linecard LCS protocol segment prepend requires that the first byte of the CRC-16 be masked when the CRC-16 is checked (see example). The following are some sample CPs and prepends, showing the CRC-16 generated for each. The CRC-16 field is always the last 16 bits. LCS Start CP: 0x0280000000000000C566 Request Count CP: 0x01037280240000006FDB LCS Prepend from Switch to Linecard: 0x8719362C09DCxxC7 1 (subport = 0) 0xE719362C09DCxxC7 1 (subport = 3, MUX masked to 0 for CRC) 0xE719362C09DCxxDF (subport = 3, MUX included in CRC) The use of the CRC-16 polynomial in an ETT1 system is the same as above, however only the last eight bits of the 16-bit CRC are valid from the Switch to the Linecard. Therefore, the receiving linecard should calculate the CRC for the entire prepend with the CRC field set to all 0's and compare the last eight bits to those of the received CRC as shown here. "xx" means these eight bits should be masked from the compare. LCS Prepend from Linecard to Switch: 0x81B9409D00106039 1 (subport = 0)
1. Note the MUX[1:0] field is assumed equal to "00" in these CRC-8 calculations.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
73
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
0xE1B9409D00106039 1 (subport = 3, MUX masked to 0 for CRC) 0xE1B9409D00103F21 (subport = 3, MUX included in CRC) The CRC-16 from the Linecard to the Switch is the same as the standard CRC-16.
1.6.4.2
Use of CRC-8 Option
Optionally, an ETT1 based system can use LCS segment prepends with the CRC-8 polynomial:
x8 + x2 + x + 1 .
1. Prior to starting the calculation, the remainder is set to 0xff. 2. In all cases, the CRC-8 is calculated over all bits (64-bits of the prepend) with the CRC field set to all zeroes. 3. Transmission of the CRC-8 is done with the most significant bit sent first. The following are some sample prepends, showing the CRC-8 generated for each. The CRC-8 field is always the last 8 bits. LCS Prepend from Switch to Linecard: 0x8719362C09DCxxFD 1 (subport = 0) 0xE719362C09DCxxFD 1 (subport = 3, MUX masked to 0 for CRC) 0xE719362C09DCxx19 (subport = 3, MUX included in CRC) The CRC-8 polynomial in an ETT1 system uses only the last eight bits of the 16-bit CRC field. Therefore, the receiving linecard should calculate the CRC for the entire prepend with the entire CRC field set to all 0's and compare the calculated CRC-8 to those of the received CRC-8 as shown here. "xx" means these eight bits should be masked from the compare. LCS Prepend from Linecard to Switch: 0x81B9409D00100096 1 (subport = 0) 0xE1B9409D00100096 1 (subport = 3, MUX masked to 0 for CRC) 0xE1B9409D00100072 (subport = 3, MUX included in CRC) The CRC-8 from the Linecard to the Switch is sent with the leading eight bits of the CRC-16 field set to zero.
1. Note the MUX[1:0] field is assumed equal to "00" in these CRC-8 calculations.
74
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.6.5
LCS PHY Layer
The LCS PHY provides a high-speed parallel transfer of data in the form of framed, fixed-sized segments that have been channelized across multiple media connections. Higher segment rates are also supported through trunking options, while lower segment rates are supported through multiplexing options. The channelization, framing, and transmission functions of the LCS PHY have been specified to allow use of a variety of available Serdes (serialization/deserialization) and parallel fiber optics components. Portions of this specification will refer to examples of media types. The LCS PHY specification does not preclude other compatible options. Channelization requirements include the mapping of segments to multiple channelized frames, channel skew specifications, and interchannel (segment) alignment operation. Frame alignment, encapsulation, retiming and the required code-points for a parallel, full-duplex interface are included in the framing function. Transmission requirements include the clocking requirements and electrical characteristics of the parallel interface. The parallel interface used by the LCS PHY is based on widely available Serdes implementations that include their own endec (encoder/decoder) functionality. These components also fit well with the available and emerging parallel fiber optics components. Optionally, an LCS device can include the Serdes and 8B/10B Endec functions, but these are not strictly required by the LCS specification. In fact, LCS implementations that forego the use of optical transceivers or Serdes and endecs altogether for single board design are also compliant with the LCS specification.
1.6.5.1
Link Speed
The LCS PHY for ETT1 supports one in link speed option, 1.5 Gbaud. This link speed has various requirements that affect channelization and framing functions, discussed here and in the LCS specification. Serdes and optical transceiver components are available to support the 1.5 Gbaud links.
1.6.5.2
Segment Payload Options
The LCS Protocol for ETT1 uses an eight byte prepend and supports two segment payload sizes, a 64-byte payload with an eight byte LCS prepend (8+64 byte segment) and a 76-byte payload with an eight byte LCS prepend (8+76 byte segment). Discussion of these options is also covered in the channelization and framing sections here and in the LCS specification.
1.6.5.3
Segment Channelization
The LCS Physical Layer includes the external interface for receiving and transmitting LCS segments from and to the linecard. The interface consists of parallel "ganged" interfaces that each carry a serialized
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
75
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
portion of a segment called a frame. An LCS linecard or switch should therefore state the number of channels in a gang and the bytes per channel utilized for its particular implementation. In the receive direction each channel receives a serial stream of 8B/10B coded data which can be decoded into a parallel byte stream plus two additional bits - one to indicate a control code-point and one for an errored code-point. In the transmit direction, an eight bit byte of parallel data plus one bit to indicate control word is sent through each channel as serialized 8B/10B encoded data. The optional parallel interface has separate ingress and egress 10-bit parallel buses. The busses operate in source synchronous mode and so carry a clock with them to latch the data. A source synchronous transmit clock is driven from the LCS device interface to the Serdes devices. For data arriving from the remote LCS device, each Serdes device recovers the receive clock from the bit stream and provides it to the local LCS device's parallel bus receiver interface. Table 15 shows the different external interface configurations supported by the ETT1. When utilizing the parallel interface, a Serdes device is used to convert between the byte-stream and the 8B/10B encoded serial stream. This Serdes will be a Single Data Rate (SDR) interface for the 1.5 Gbit/s interface.
Table 15.
Gang of Channels 12 14 Data Frame Size 6 6
Fiber 1.5 Gbit/s per Channel Interface Configurations
Segment Size 72 84 Segment Time 40 ns 40 ns Segments per second 25 M 25 M Channel Data Rate 1.50 Gbits/s 1.50 Gbits/s Channel Line Rate 1.5 SDR 1.5 SDR
Table 16.
Gang of Chnls 14 Data Frame Size 6 Channel 1 Prepend 0-5 Channel 2 Prepend 6-7, Pyld 0-3 Prepend 6-7, Pyld 0-3
Mapping Segments to 1.5 Gbit/s Channels
Channel 3 Payload 4-9 Channel 4 Payload 10-15 Channel 8 Payload 34-39 Channel 9 Payload 40-45 Channel 12 Payload 58-63 Channel 14 Payload 70-75 Channel 15
12
6
Prepend 0-5
Payload 4-9
Payload 10-15
Payload 34-39
Payload 40-45
Payload 58-63
An LCS segment consists of an eight byte prepend and either a 64- or 76-byte payload. The start of the prepend is sent on the first channel, any remainder of the prepend is sent on the next channel, followed by the start of the payload. The remaining channels carry the subsequent bytes of the payload. The exact mapping of bytes to channels is shown Table 16. An 8+64 byte segment is not explicitly supported when using the 2.5 Gbaud links. However, any smaller segment payload than 76 bytes can be supported by padding the fixed length segments to the full 76-byte payload.
76
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.6.5.4
Alignment, Framing, and Transmission Functions
Details of the alignment, framing, and transmission function of the LCS Protocol are covered in the "LCS Protocol Specification -- Protocol Version 2", available from PMC-Sierra, Inc.
1.7
THE OUT-OF-BAND (OOB) BUS INTERFACE
The Out-of-Band (OOB) bus provides a mechanism for a local CPU to control some or all ETT1 devices within a switch fabric. The OOB bus provides about 1Mbyte/sec of bandwidth which should be sufficient for the intended task. The OOB is a "local" bus in the sense that it only interconnects devices that are co-located on the same printed circuit board. A complete fabric will consist of many boards and therefore many separate OOB buses. The multiple OOB buses could be extended into a single logical bus via the use of a "satellite" bus controller on each board as shown in Figure 32. Alternatively, each OOB bus could be controlled by a local CPU and the CPUs could then communicate with the other CPUs via some other mechanism such as an Ethernet network. This document describes only the "local" OOB bus. All of the ETT1 devices have an OOB interface. The ETT1 devices are slave devices and so there must be a local master device, such as a satellite controller, that initiates bus transactions.
Figure 32. One Implementation: The OOB Bus Consists of Both Local and Global Buses
PCB ETT1 ASIC Satellite Controller ETT1 ASIC local OOB bus global OOB bus PCB ETT1 ASIC Satellite Controller ETT1 ASIC local OOB bus (master) Central Controller CPU (master) PCB
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
77
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.7.1
The OOB Bus
The OOB bus is an eight-bit, tri-state, multiplexed address/data bus with separate control and interrupt signals.
1.7.1.1
OOB Bus Signals
Thirteen signals are used:
Table 17. OOB Control Bus Signals
Signal Name AD[7:0]
Description 8 bit address/data bus. AD[7] indicates the type of cycle (1 = Write, 0 = Read) during the first address cycle, and is used for for address/data during other cycles. A[6:0] must provide the board select address when VALID_L is high. Bus cycle is valid. Active low. Wait. Active low. Initially asserted to indicate slave will respond. Then asserted, if necessary, to allow slave extra cycles to respond to current transaction. Clock. High priority interrupt Low priority interrupt
Driven by M(aster) or S(lave) Both. Address driven by master. Data supplied by master for a write cycle. Data supplied by Slave for a read cycle. Master.
VALID_L WAIT_L
Slave
CLK INT_HI INT_LO
Clock source. Slave Slave
NOTE: Active low signals are indicated by "_L". All signals are point to point except for the AD bus which is a multipoint bus. In particular, WAIT_L and the interrupt signals are not open collector/gate drivers.
1.7.1.2
Address Space
The OOB bus provides 31 bits of address - A[30:0]. Of these, bits A[1] and A[0] are always 0 as the bus does not support 8- or 16-bit accesses: all accesses are 32-bits wide, and must be aligned on quad byte boundaries.
78
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Bit A[31] is used to indicate the cycle type: Write or Read. The remaining 31 bits are divided into a device select block and a sub-address block.
Global Device Select (11b) 30 24 23 16 15 Sub-address (20b) 8 7 00 0
The global device select block is 11 bits and is used to identify a target device. The sub-address is used by the target device to identify a register or memory location within the device. The device select block is divided into three categories as follows: 0x7FF Broadcast Write. All devices should respond to this device select value but will choose to ignore the write if they do not support a broadcast write to the specified sub-address. All devices assert WAIT_L until their local write operation completes. Multicast device selects. (15 multicast groups). See Table 18 below for groups. Individual device selects. See Table 19 for details of one possible address scheme.
0x7F0 to 0x7FE 0x0 to 0x7EF
The current multicast groups are shown in Table 18.
Table 18. Multicast Group Selects
Group All EPPs All Dataslices All Schedulers All Crossbars All Satellites (Not part of the ETT1 Chip Set)
Device select value (11bits) 0x7FE 0x7FD 0x7FC 0x7FB 0x7F0
1.7.1.3
Individual Device Selects
The individual device selects (30:20) are hierarchical: the most significant seven bits (30:24) are the board address and identify a single board (PCB) in the system; the four least significant bits (23:20) determine which particular device on a board is being selected. In addition, the two most significant bits (30:29) indicate a board type: Port board, Crossbar board or Scheduler board. Figure 33 shows this organization.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
79
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 33. The Global Device Select Defines the Board Number and the Device on Each Board
Global Device Select (11b) 30 24 23 16 15
Sub-address (20b) 8 7 0
Board address 30 Board type: 00 = Port 01 = Crossbar 10 = Scheduler 24 23
Local Device Select
20
At power-on the ETT1 devices do not know which board they are on. They learn the board address (bits 30:24) by continuously observing the OOB bus itself: every board must have an OOB controller which must drive the unique board address on AD[6:0] whenever VALID_L is not active. The OOB controller must continue to do this while the system is active. Each port board (and EPP) obtains its port number by looking at AD[4:0] when VALID_L is not active, which is why AD[6:0] must be valid whenever VALID_L is inactive. Each device obtains its own local device select (bits 23:20) either from static pins on the device, or through a preset address. For the Enhanced Port Processor, Crossbar, and Scheduler devices, four dev_sel pins (oobdev_sel[3:0]), are tied to logical 1 or 0 to provide the four least significant bits of the unique device address. For the Dataslice device, three dev_sel pins (oobdev_sel[2:0]) are tied to logical 1 or 0 to provide three of the four least significant bits of the unique device address. Since there are two logical Dataslices within each package, the fourth implied bit is used to select which of the two logical Dataslices is addressed.
}
80
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
The combinations of permissible board addresses and device addresses are shown in Table 19. NOTE: If the satellite OOB controller needs its own address within the OOB space then it can use 0xPPe (i.e. local device select = 0x00e).
Table 19. Unicast Group Selects
Devices EPP on port P (P=0..31) Dataslice D (D=0..13) on port P (P=0..31) Scheduler S (S=0,1) Crossbar C (C=0..31)
Address Format (bin) {00P_PPPP_1111} {00P_PPPP_DDDD}a {100_000S_0000} {010_CCCC_000C}b
Address range allocated 0x00f, 0x01f, ... 0x1ff 0x000-0x00d,..0x1f0-0x1fd 0x400 or 0x410 0x200, 0x201, 0x210...0x2f1
a. In the ETT1 reference system design, P_PPPP corresponds to the port number (0-1f hex) and DDDD corresponds to the Dataslice number (0-d hex) b. In the ETT1 reference system design, the first four bits of the CCCC_000C designation correspond to the crossbar board number, and the last bit of the CCCC_000C designation specifies which of the two crossbar chips on that board is being addressed. Note that in the reference design, logical crossbars 0 and 6 are on the board with CCCC=0000, logical crossbars 1 and 7 are on the board with CCCC=0001, etc.
1.7.1.4
Port Board Assignment
The port boards must have unique board select addresses in the range 0 to 31. These addresses identify the logical port number of a port for LCS purposes. The assignment of port addresses is not random and must reflect the physical connections made between the port boards and the crossbar boards. For example, one port board will have connections to port 6 on every Crossbar device and port 6 on every Scheduler. That port board must then have a board address of 6.
1.7.1.5
Bus Cycles
Two types of bus cycles are supported: write and read. Each type may have any number of wait states before the cycle completes.
1.7.1.6
A Write Cycle - No Wait
A simple 32-bit write. The master drives out a 31-bit address, 8 bits at a time, most significant bits first; AD[7] is high to indicate a write operation. VALID_L is asserted by the master to indicate the new cycle. NOTE: The target slave asserts WAIT_L during the final address cycle to indicate that the target is active. The target then de-asserts WAIT_L with the first data cycle. The master must de-assert VALID_L for at least one cycle before re-asserting it for the next cycle. Whenever VALID_L is de-asserted the master must drive the local board address onto AD[6:0].
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
81
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 34. Write Cycle - No Wait
A0 CLK AD[7] AD[6:0] VALID_L WAIT_L W/R board address a30:24 a23 a22:16 a15 a14:8 a7 a6:0 d31 d30:24 d23 d22:16 d15 d14:8 d7 d6:0 board address A1 A2 A3 D0 D1 D2 D3 Rec Idle
Addresses are presented during cycles A0-A3. The write data is presented during D0-D3. There is then a recover cycle during which the target performs the write.
1.7.1.7
A Write Cycle - One Wait
If, during D3, the target decides that it may not be able to complete the write immediately, then it must assert WAIT_L. It continues to assert WAIT_L until the write is completed or WAIT_L has been asserted for 256 cycles, at which time WAIT_L is de-asserted. (If WAIT_L is asserted for 256 cycles then the master will consider this to be a bus error). Figure 35. shows a write cycle with the target forcing a one cycle wait. The target slave asserts WAIT_L during A3 to indicate that the target is active. The target then de-asserts WAIT_L and re-asserts it during D3. The master must maintain d7:0 until WAIT_L is de-asserted.
Figure 35. Write Cycle - One Wait
A0 CLK AD[7] board AD[6:0] address VALID_L WAIT_L W/R a30:24 a23 a22:16 a15 a14:8 a7 a6:0 d31 d30:24 d23 d22:16 d15 d14:8 d7 d6:0 board address A1 A2 A3 D0 D1 D2 D3 Wt Rec
1.7.1.8
A Read Cycle - One Wait
A simple 32-bit read. The master drives out a 31-bit address, 8 bits at a time, most significant bits first; AD[7] is driven low in the first cycle to indicate a read cycle. VALID_L is asserted by the master to indicate the new cycle. The target slave asserts WAIT_L during A3 to indicate that the target is active. The target
82
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
keeps WAIT_L asserted through bus turn-around cycle (effectively a wait cycle). The target de-asserts WAIT_L once the first valid data byte is on the bus. VALID_L must be high for at least two rising clocks before the next cycle. There is an additional bus turn-around cycle during the Recover cycle. This is the fastest read cycle: there is always at least one bus turn-around cycle whenever AD is tri-stated. The master must drive the AD signals before the next rising edge of the clock after the VALID_L is deasserted.
Figure 36. Read Cycle - One Wait
A0 CLK AD[7] W/R board AD[6:0] address a30:24 VALID_L WAIT_L a23 a22:16 a15 a14:8 a7 a6:0 d31 d30:24 d23 d22:16 d15 d14:8 d7 d6:0 board address A1 A2 A3 Wt D0 D1 D2 D3 Rec Idle
1.7.1.9
A Read Cycle - Two Waits
This is the same as a single wait read cycle except that there is an extra wait cycle before D31:24 are valid and WAIT_L is de-asserted.
Figure 37. Read Cycle - Two Waits
A0 CLK AD[7] W/R board AD[6:0] address a30:24 VALID_L WAIT_L a23 a22:16 a15 a14:8 a7 a6:0 d31 d30:24 d23 d22:16 d15 d14:8 d7 d6:0 board address A1 A2 A3 Wt Wt D0 D1 D2 D3 Rec Idle
1.7.1.10
Interrupts
Two active high interrupt lines are provided. INT_HI is for "emergency" interrupts such as a card failing. INT_LO is for non-emergency interrupts such as a device requesting to be polled for some reason. Interrupts are asserted and de-asserted synchronously with respect to the clock. Interrupts can be asserted at any time, even during an access.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
83
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
The bus master must clear an interrupt by reading an appropriate register within the device that has asserted the interrupt.
Figure 38. Interrupts are Active High and Synchronous
CLK INT_HI INT_LO
Bus Rules 1. The target slave must assert WAIT_L during the final address cycle. If the master does not see WAIT_L asserted during the final address cycle then it concludes that there is no device at that address and terminates the cycle (takes VALID_L high). 2. VALID_L must be de-asserted (high) for at least one clock cycle between accesses. The master must drive the board address on AD[6:0] during a positive clock edge while VALID_L is de-asserted (high). 3. A slave may insert up to 255 wait states before it must de-assert WAIT_L. Failure to de-assert WAIT_L within that time will result in a Bus Error. (The master will terminate the cycle by de-asserting VALID_L)
1.8
INITIALIZATION PROCEDURE
Every ETT1 component must be correctly initialized before being used within a switch system. In general, the process of initialization depends on whether the whole switch is being initialized (initial power-on sequence) or whether a component is being added to a switch system that is already operating. However, to simplify the process, this document describes an initialization sequence that can be used in either case, but does incur some redundant operations when used at system initialization. The initialization sequence is also a function of the physical organization of the ETT1 devices. The sequence described here assumes the physical configuration that has been used in the ETT1 Reference System: each port board has one Enhanced Port Processor and six or seven Dataslice devices; each Crossbar board has two Crossbar devices; each Scheduler board has one Scheduler device. The system has redundant Crossbars and Schedulers. The link between the linecards and the ETT1 ports uses industry standard Serdes devices. NOTE: For more information on the link synchronization involving the Serdes and the fiber optics, see Appendix C, Section 2 "The 8b/10b Interface to the Dataslice" on page 328.
84
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.8.1
* *
Initial Power-on:
Wait for voltage ramp-up Suggested:
*
* * * * * * * * * *
Initialize the OOB satellite FPGA
Reset Enhanced Port Processor Reset Dataslices Reset Scheduler Reset Crossbar Enable PLLs on all devices (turn off PLL reset) Program Dataslices' DINBY and DOUTBY to 3 Set interrupt masks (disable all except AIB ready up and external link interrupts) on EPP & DSs Set primary Crossbar in DSs to one that exists Program 8b10b tables of DSs Program pre-defined switch to Linecard Control Packets location in all DSs. ( See section 3.3, table 28) EPP Waiting Scheduler Request Count Memory EWSRCT address offset 1C000-1DFFCh: After resetting an EPP, write 00000000h to address offsets 1D000h through 1D7FCh in that EPP (a total of 512 OOB writes). Whan all EPPs are resetsimultaneously (using a broadcast OOB write to the EPP Reset and Control register), 512 broadcast OOB writes can be used to reset those locations in all EPPs. Release VCSEL and PIN diode resets on DSs Suggested:
*
* *
* * * *
*
enable clock phase shifting on Serdes, configure Serdes loopback controls for normal operation, enable Serdes synchronization, send idle-not-rdy from linecard
Enable DS 8b10b encoder
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
85
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
* *
Enable DS 8b10b decoder Suggested:
*
* *
disable Serdes clock phase shifting when sync is established
Set Scheduler BP depth field of Control register to 0x14. Set Scheduler action bits (Configuration dependent. See "Enhanced Port Processor - Scheduler" on page 83) Enable the Flow Control Crossbar sync from the Scheduler Reset DS AIB (for all ports present) Reset Crossbar AIB (for all ports present)(includes Flow Control Crossbars) Reset EPP AIB (for all ports present) Reset Scheduler AIB (for all ports present) Enable DS AIB (for all ports present) Enable Crossbar AIB (for all ports present)(includes Flow Control Crossbars) Release reset on DS AIB Release reset on Crossbar AIB Release reset on EPP AIB Release reset on Scheduler AIB
* * * * * * * * * * *
These sequences should complete with only the "ready up" interrupts enabled. When these interrupts occur (which may be during later parts of the above sequences) then perform the following: When DS AIB ready up: * * Inform local processes that the DS is operational Enable all interrupts on DS, except for Transceiver Comma Detect and 8b10b Decoder Comma Detect which are set every time an idle is received.
When EPP AIB ready up for at least one Scheduler and at least one set of Flow Control Crossbars: * * Inform local processes that the EPP is operational Enable all interrupts on EPP
86
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
* * * *
Set Fault Tolerance Control Enable Port Processor Define 1-in-N Idle Count value Take EPP out of LCS Stop mode
When Crossbar AIB ready up: * * Inform local processes that the Crossbar is operational Enable all interrupts on Crossbar
When Scheduler AIB ready up: * * * * * Inform local processes that the Scheduler is operational Enable all interrupts on Scheduler Enable ports that are present and up Enable non-TDM traffic If TDM traffic, define TDM control and Enable TDM traffic
In a system with redundant Schedulers, it is essential to perform a refresh operation as the final step, once ports are configured and operational. This synchronizes the two Schedulers. Refresh schedulers if necessary (do not refresh if only one in system)
1.8.2
* *
Port Board being added:
Wait for voltage ramp-up Suggested:
*
*
Initialize the OOB satellite FPGA
If a Scheduler is present and has its Control register's CRC Action bit set, save and clear the CRC Action bit. Reset Enhanced Port Processor Reset Dataslices Enable PLLs on EPP and DSs (turn off PLL reset)
* * *
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
87
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
* * * * *
Program Dataslices' DINBY and DOUTBY to 3 Set interrupt masks (disable all except AIB ready up and external link interrupts) on EPP & DSs Set primary Crossbar in DSs to one that exists Program 8b10b tables of DSs Program pre-defined switch to Linecard Control Packets location in all DSs. ( See section 3.3, table 28) EPP Waiting Scheduler Request Count Memory EWSRCT address offset 1C000-1DFFCh: After resetting an EPP, write 00000000h to address offsets 1D000h through 1D7FCh in that EPP (a total of 512 OOB writes). Whan all EPPs are resetsimultaneously (using a broadcast OOB write to the EPP Reset and Control register), 512 broadcast OOB writes can be used to reset those locations in all EPPs. Release VCSEL and PIN diode resets on DSs Suggested:
*
* *
* * * *
* * *
enable clock phase shifting on Serdes, configure Serdes loopback controls for normal operation, enable Serdes synchronization, send idle-not-rdy from linecard
Enable DS 8b10b encoder Enable DS 8b10b decoder Suggested:
*
disable Serdes clock phase shifting when sync is established
If neighboring Crossbar cards are present: * * * Reset DS AIB and Crossbar AIB Enable DS AIB and Crossbar AIB Release reset on DS AIB and Crossbar AIB
If neighboring Flow Control Crossbar cards are present: * Reset EPP AIB and Flow Control Crossbar AIB
88
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
* *
Enable EPP AIB and Flow Control Crossbar AIB Release reset on EPP AIB and Flow Control Crossbar AIB
If neighboring Scheduler cards are present: * * * * Reset EPP AIB and Scheduler AIB Enable EPP AIB and Scheduler AIB Release reset on EPP AIB and Scheduler AIB Restore the Scheduler Control register CRC Action bit if it was set before initializing this Port Board.
These sequences should complete with only the "ready up" interrupts enabled. When these interrupts occur (which may be during later parts of the above sequences) then perform the following: When DS AIB ready up: * * Inform local processes that the DS is operational Enable all interrupts on DS, except for Transceiver Comma Detect and 8b10b Decoder Comma Detect which are set every time an idle is received.
When EPP AIB ready up for at least one Scheduler and at least one set of Flow Control Crossbars: * * * * * * Inform local processes that the EPP is operational Enable all interrupts on EPP Set Fault Tolerance Control Enable Port Processor Define 1-in-N Idle Count value Take EPP out of LCS Stop mode
1.8.3
* *
Scheduler Board being added:
Wait for voltage ramp-up Suggested:
*
Initialize the OOB satellite FPGA
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
89
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
* * * *
Reset Scheduler Enable PLLs on SCH (turn off PLL reset) Set BP depth field of Control register to 0x14. Set Scheduler action bits (Configuration dependent. See "Enhanced Port Processor - Scheduler" on page 83) Enable the Flow Control Crossbar sync Set interrupt masks (disable all expect ready up)
* *
If neighboring Port cards are present: * * * * Reset EPP AIB and Scheduler AIB (for all ports present) Enable EPP AIB and Scheduler AIB (for all ports present) Release reset on EPP AIB and Scheduler AIB (for all ports present) Reset ports on Scheduler
These sequences should complete with only the "ready up" interrupts enabled. When these interrupts occur (which may be during later parts of the above sequences) then perform the following: When Scheduler AIB ready up: * * * * * Inform local processes that the Scheduler is operational Enable all interrupts on Scheduler Enable ports that are present and up Enable non-TDM traffic If TDM traffic, define TDM control and Enable TDM traffic
In a system with redundant Schedulers, it is essential to perform a refresh operation as the final step, once ports are configured and operational. This synchronizes the two Schedulers. Refresh schedulers if necessary (do not refresh if only one in system)
1.8.4
* *
Crossbar Board being added:
Wait for voltage ramp-up Suggested:
90
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
*
* * *
Initialize the OOB satellite FPGA
Reset Crossbar Enable PLLs on Crossbar (turn off PLL reset) Set interrupt masks (disable all except ready up)
If neighboring Port cards are present and this is a Flow Control Crossbar: * * * Reset EPP AIB and Flow Control Crossbar AIB (for all ports present) Enable EPP AIB and Flow Control Crossbar AIB (for all ports present) Release reset on EPP AIB and Flow Control Crossbar AIB (for all ports present)
If neighboring Port cards are present and this is a data Crossbar: * * * Reset DS AIB and Crossbar AIB (for all ports present) Enable DS AIB and Crossbar AIB (for all ports present) Release reset on DS AIB and Crossbar AIB (for all ports present)
These sequences should complete with only the "ready up" interrupts enabled. When these interrupts occur (which may be during later parts of the above sequences) then perform the following: When Crossbar AIB ready up: * * Inform local processes that the Crossbar is operational Enable all interrupts on Crossbar
1.9
FAULT TOLERANCE
A ETT1 fabric can optionally be configured with redundant elements. When the ETT1 system contains redundant Schedulers and Crossbars, it is capable of sustaining certain faults and yet not lose or corrupt any of the cells within the fabric.
1.9.1
Fault Tolerance Model
The fault tolerance model assumes that any failure within a redundant element will manifest itself as a link error, and thus will be observable by the local CPU. The redundant elements are the Scheduler and Crossbar which are connected via AIB links to the Enhanced Port Processor and Dataslice, as shown in Figure 39. These AIB links are protected by a CRC such that a failure in a device will likely result in a CRC error, which can be detected by the local CPU through interrupts. Furthermore, even if a CRC error is not
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
91
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
generated, the Enhanced Port Processor and Dataslice will detect any difference in the information received from the two, supposedly identical elements and will act as though an error had occurred.
Figure 39. Redundant Connections
Crossbar 0
Dataslice Crossbar 1
Dataslice
EPP Flow Control Crossbar 0 Port Flow Control Crossbar 1
EPP
Port
Scheduler 0
Scheduler 1
Fault tolerant region
In this section, we consider two types of errors that may occur: soft errors and hard errors. Soft errors are transient errors that do not require replacement of any of the boards. Hard errors are caused by a device failure resulting in a defective board which should be replaced. On the AIB links a soft error would appear as a transient CRC error; a hard error would be indicated by the link ready signal going inactive (see Section 1.10.3 "AIB Links (A)" on page 103). The significance of this fault model is that all faults associated with redundant elements will appear as a link error of some form. So the fault detection and recovery procedures for any ETT1 fabric can be
92
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
specified by considering the various AIB links. The following sections explain these procedures for redundant and non-redundant configurations.
1.9.2
Soft/Transient Errors
Intermittent CRC errors between the Dataslice and Crossbar and the Enhanced Port Processor and Scheduler are considered in this section. The corrective action that needs to take place due to these errors are different for non-redundant and redundant configurations.
1.9.2.1
Non-Redundant Configurations
Dataslice - Crossbar In a non-redundant Crossbar configuration, CRC errors that occur between the Dataslice and Crossbar imply that part of a cell has been corrupted. In Figure 40, Dataslice 1 and Dataslice 2 are connected to Crossbar 0. If a CRC error occurs from Dataslice 1 to Crossbar- 0 when valid data is to be transferred, then that slice of information for that cell has been corrupted. As a result, Dataslice 2 will receive an invalid cell. Crossbar 0 will indicate to the CPU that a CRC error has occurred. If Dataslice 2 receives a CRC error from Crossbar-0 when valid data is to be transferred, then the cell has been corrupted. Dataslice 2 will indicate to the CPU that a CRC error has occurred. In both of the above, if the corrupted cell did contain cell data (as opposed to being an empty cell) Dataslice 2 will store the corrupted information in its local memory and forward it to the egress linecard. The ETT1 fabric will not make the decision to discard the cell; the linecard must detect the error and discard the cell. The linecard would detect such an error with it's own payload error detection scheme. If the CRC error occurred in an empty cell then no information is lost. However the occurrence of a CRC error is always indicated to the local CPU via a maskable interrupt.
Figure 40. Simple Non-Redundant Crossbar Configuration
Crossbar 0 Dataslice 1 Dataslice 2
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
93
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Enhanced Port Processor - Flow Control Crossbar If CRC errors occur between the Flow Control Crossbar device and the EPP, the input EPP will not have the correct view of the available space in the egress queues of the output EPP. Thus, a refresh of the output queue credits must be performed. The invalid incremental credit interrupt register is used to indicate which input port to output port flow has lost output queue credits. For example, if port 5's invalid incremental credit interrupt register has a value of 0x400, then there are lost credits for all Unicast flows from input port 5 to output port 10. See Section 1.9.3 "Flow Control Refresh Procedure" on page 97. Enhanced Port Processor - Scheduler In a non-redundant Scheduler configuration, CRC errors that occur between the Enhanced Port Processor and Scheduler might result in inconsistent state information. The Scheduler must have accurate information on the number of outstanding requests as well as the backpressure state of all VIQs. A lost request or grant can cause a discrepancy between the Enhanced Port Processor and Scheduler. Figure 41 describes a simple non-redundant Scheduler configuration where Enhanced Port Processor 1 and Enhanced Port Processor 2 are connected to Scheduler 0. If Enhanced Port Processor 1 receives a CRC error on information from Scheduler 0, then Enhanced Port Processor 1 does not know if that information contained a valid grant. If the corrupted grant was for unicast traffic, then the Enhanced Port Processor would have one more request than the Scheduler; this might delay the forwarding of one cell. However, if the corrupted grant was for multicast traffic, then the next multicast grant might cause a cell to be sent to the wrong destination port. This is considered unacceptable and so a CRC error from Scheduler-0 makes Enhanced Port Processor 1 ignore all subsequent Scheduler grants until the state information can be restored. As always, the CRC error is indicated to the local CPU which must refresh Scheduler 0 before traffic can continue to flow to/from this port. The Enhanced Port Processor continues to receive routing tags from the Scheduler, and will forward cells received from other ports. If Scheduler 0 receives a CRC error on information from Enhanced Port Processor- , then Scheduler 0 may have lost a new request or backpressure information. If the lost information was a unicast request, then the Scheduler would have one less request than the Enhanced Port Processor. This might delay the forwarding of one cell. If the lost information was a multicast request, the Scheduler might send a cell to the wrong destination port. If back pressure assertion was lost, the Scheduler could possibly overflow the output queues. These behaviors are considered unacceptable. Thus, when Scheduler 0 detects a CRC error it disables traffic to/from Enhanced Port Processor 1 until the CPU refreshes Scheduler 0's state information. Traffic to/from other ports continues to flow. The CRC error is indicated to the CPU which must then refresh the state information in Scheduler 0. See Section 1.9.4 "Scheduler Refresh Procedure" on page 98.
94
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 41. Simple Non-Redundant Scheduler Configuration
Enhanced Port Processor 1
Enhanced Port Processor 2
Scheduler 0
1.9.2.2
Redundant Configurations
For a redundant Crossbar configuration, the redundant connection provides the uncorrupted information between the Dataslice and Crossbar. In Figure 42, Dataslice 1 and Dataslice 2 are connected to both Crossbar 0 and Crossbar 1. The same information is sent from the Dataslices to each of the Crossbars, and therefore, the same information is received by the Dataslices from each of the Crossbars. Consider when a CRC error occurs from Dataslice 1 to Crossbar 0, and no CRC error occurs from Dataslice 1 to Crossbar 1. In this situation, Dataslice 2 will receive invalid information from Crossbar 0, but will receive valid information from Crossbar 1. Dataslice 2 uses the valid information and no cells are corrupted. Crossbar 0 will indicate to the CPU that a CRC error has occurred. If a CRC error occurs from Crossbar 0 to Dataslice 2, Dataslice 2 would not receive valid information from Crossbar-0. Since Crossbar 1 receives the same information as Crossbar 0, Dataslice 2 will get the valid information from Crossbar 1. Dataslice-2 uses the valid information and no cells are corrupted. Dataslice 2 will indicate to the CPU that a CRC error has occurred. In both situations, the simple redundant Crossbar configuration has prevented cell loss in the event of a single CRC error. The occurrence of a CRC error between the Dataslice and Crossbar is always noted, but no corrective action is required.
Figure 42. Simple Redundant Crossbar Configuration
Crossbar 0 Dataslice 1 Crossbar 1 Dataslice 2
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
95
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Enhanced Port Processor - Flow Control Crossbar The redundant configuration is equivalent to that of the Dataslice to Crossbar configuration: A single CRC error on one of the links can be ignored because the EPP will automatically use the valid information from the other link. However, if the EPP receives four consecutive CRC errors on its primary FC crossbar link then it will consider the link to have failed, and will make the other link the new primary link. See the EFTCTRL register in the EPP for more details. If CRC errors occur on both links during the same cell time then information will be lost, and the recovery process described in the non-redundant configuration must be used to restore the correct state in the input EPP. Enhanced Port Processor - Scheduler In a redundant Scheduler configuration, the redundant connection provides the uncorrupted information between the Enhanced Port Processor and Scheduler. Figure 43 describes a simple redundant Scheduler configuration where Enhanced Port Processor 1 and Enhanced Port Processor 2 are connected to both Scheduler 0 and Scheduler 1. In this example, Scheduler 0 is the primary Scheduler and Scheduler 1 is the secondary Scheduler.
Figure 43. Simple Redundant Scheduler Configuration
Enhanced Port Processor 1
Enhanced Port Processor 2
Scheduler 0 (primary)
Scheduler- 1 (secondary)
If, due to an error, the two Schedulers send different information to the EPPs then it is clearly essential that all of the EPPs must use the information from the same Scheduler. Therefore, all of the Enhanced Port Processors must use a consistent setting for their primary/secondary Scheduler, which is defined in the EFTCTRL register in the EPP. The corrective action taken is different for CRC errors to/from the primary Scheduler and CRC errors to/from the secondary Scheduler. These are described below. If Enhanced Port Processor 1 receives a CRC error on information from Scheduler 0 (primary), then it uses the uncorrupted grant information from Scheduler 1. Similarly, if Enhanced Port Processor 1 receives a CRC error on information from Scheduler 1, then it uses the information from Scheduler 0. In both these case, the CPU is notified of the error, but no corrective action is required by the CPU since the state information remains consistent. However if the EPP sees four consecutive CRC errors on the link from its
96
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
current primary Scheduler then it will automatically consider the other Scheduler as the new primary Scheduler. If Scheduler 0 (primary Scheduler) receives a CRC error on information from Enhanced Port Processor 1, then Scheduler 0 and Scheduler 1 might differ in their state information. In this situation Scheduler 0 immediately disables its AIB links to Enhanced Port Processor 1 and Enhanced Port Processor 2 (and all Enhanced Port Processors). The Enhanced Port Processors immediately detect the loss of connection to Scheduler 0 and automatically select Scheduler 1 (secondary Scheduler) to be their new primary Scheduler. Without CPU intervention, the system has disabled the primary Scheduler, and switched the Enhanced Port Processors to use the secondary Scheduler. The CPU is informed of this error, and must undertake the process to refresh the Schedulers.The EPP's may register Scheduler mismatch interrupts until the Schedulers are refreshed. The AIB links to other EPPs are not affected. If Scheduler 1 (secondary Scheduler) receives a CRC error on information from Enhanced Port Processor 1, then Scheduler 1 immediately disables its AIB links to Enhanced Port Processor 1. The Enhanced Port Processors do not alter their primary/secondary setting, since Scheduler 0 is already selected as the primary Scheduler. The CPU is notified of this error, and must go through the steps to refresh the Schedulers. The actions the Scheduler performs upon detecting a CRC error are different in the redundant and non-redundant configurations. Consequently the Scheduler must be configured to be in redundant or non-redundant mode. This configuration is accomplished via the CRC Action bit in the Scheduler's control register (SCTRLRS). The CRC Action bit should only be set to 1 for the primary Scheduler in a redundant configuration.
1.9.3
Flow Control Refresh Procedure
The previous section refers to the process of refreshing flow control state. The purpose of this procedure is to insure consistency of the state information between the iEPPs and the oEPPs. In the non-redundant Flow Control Crossbar configuration, the refresh procedure ensures that iEPPs will have correct queue depth information about oEPPs The refresh procedure is as follows: 1. Detect invalid incremental credit interrupt on port i. Read invalid incremental credit interrupt register on port i. The value specifies to which output port j flow could have missing output queue credits. 2. Write freeze unicast Output Scheduler flows register on port j with a value of 1 << i. This will cause the Output Scheduler on port j to stop sending cells from port i. 3. Disable the non TDM enable for port i in the Scheduler. 4. Freeze the Scheduler Request Modulator on port i. 5. Read the output unicast queue information memory (queue length) on port j for queues corresponding to input port i. If port j is in OC-192c mode, then 4 queue lengths are stored. If port j is in OC-48c mode, then 16 queue lengths are stored. 6. Read the waiting scheduler request count memory on port i for queues corresponding to output
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
97
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
port j. The number of request counts stored should be the same as above. 7. Add the stored queue length with the waiting scheduler request count and write that value into the output unicast queue debit count memory for each corresponding i,j output queue. The number of write accesses should be 4 in OC-192c mode and 16 in OC-48c mode. 8. At this point in time, the output unicast queue debit count memory is up to date, and the unicast Output Scheduler can be unfrozen, the non TDM traffic enabled, and the scheduler request modulator unfrozen.
1.9.4
Scheduler Refresh Procedure
A previous section refers to the process of refreshing a Scheduler. The purpose of this procedure is to ensure consistency of the state information between the Enhanced Port Processors and the Scheduler(s). In the redundant scheduler configuration, the refresh procedure ensures that the two Schedulers will contain identical state information and make the same scheduling decisions. The refresh procedure is as follows: 1. Disable non-TDM traffic in the Scheduler(s): set the enable non-TDM traffic register (SNTDMEN, 0x80) to 0x0. Do not modify the Enable Port register (SENBPRT,0x38). Do not modify the TDM traffic register. 2. Issue the "Go Refresh" command to the Enhanced Port Processors. The EPPs then send their state information to the Scheduler(s). 3. Wait until the refresh process has completed: for each port that issued a "Go Refresh" command, wait until the appropriate bit in the Scheduler's `Port is Refreshed' register (SRFRS, 0x94) is set to 1. 4. Enable non-TDM traffic in the Scheduler(s): set the enable non-TDM traffic register (SNTDMEN, 0x80) to 1 for each port that is enabled. NOTE: An ETT1 Scheduler device will not schedule best-effort traffic until a "Go Scheduler" command is sent from all of the Enhanced Port Processors. NOTE: Extra writes are required at this point. Refer to the "Scheduler Device Errata, issue 3" document, available from PMC-Sierra, Inc. 5. Issue the "Go Scheduler" command to all of the Enhanced Port Processors that issued "Go Refresh". This command synchronizes the two Schedulers to start scheduling best-effort traffic at the same time. 6. If the EPP's EFTCTRL register's "Ignore Central Scheduler Grants" or "Freeze SRM" bits are set, clear these bits. 7. The Scheduler(s) now contain the same state information as the EPPs and will start scheduling cells. In the redundant Scheduler configuration, the Schedulers will also make identical scheduling decisions.
98
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.9.5
Refresh Schedulers After Modifying Registers
The CPU cannot guarantee to modify the same register in both Schedulers at exactly the same instant, even using an OOB multicast write to all Schedulers. Consequently there are certain Scheduler registers which, when modified, require a refresh process to occur (assuming a system with two Schedulers). Table 20 list these registers.
Table 20. Refresh-sensitive Registers in the Scheduler
Address 00004h 00038H 00080h 00084h 00088h 0008Ch 0009Ch 00100h
Access Read/Write Read/Write Read/Write Read/Write Read/Write Read/Write Read/Write Read/Write Control and Reset Enable Port
Register
Symbol SCTRLRS SENBPRT SNTDMEN STDMEN SPORTRS SPORTFRZ STDMCTRL SPLL
Enable non-TDM Traffic Enable TDM Traffic Reset Port Freeze Port TDM Control PLL control/status
1.9.6
Hard Errors
This section describes the sequence of events that need to take place to correctly replace a failed board. When a board is replaced, it must be resynchronized with an already running system. In this section, we address each type of board replacement and describe how to resynchronize the board.
1.9.6.1
Port Board
When a port board fails, the Schedulers must disable traffic flowing to/from this port. All the cells currently stored in the input and output queues of the failed port board will be lost. These are the following steps required to replace the failed port board. NOTE: The failure of the port board may also cause the primary Scheduler to shut down. 1. Detect failed port board. 2. Disable TDM and best-effort traffic in the Scheduler(s) to/from this port. 3. Disable this port in the Scheduler's SENBPRT register. 4. Disable the AIB links to/from the Crossbars and Schedulers. 5. Replace failed port board.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
99
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
6. Bring up new board. See "Port Board being added:" on page 87. 7. Enable TDM traffic in Scheduler for this port. TDM Traffic can now flow to/from this new port board. 8. Refresh state information from this new port to Schedulers. (See Section 1.9.4 "Scheduler Refresh Procedure") This is only necessary if the linecard sends traffic before the link to the Scheduler board comes up. Re-establish primary and secondary Schedulers in every port. 9. Best-effort traffic can now flow to/from this new port board.
1.9.6.2
Crossbar Board
The steps needed to replace a failed Crossbar board on a non-redundant Crossbar configuration and a redundant Crossbar configuration are different. For a non-redundant Crossbar configuration, a failed Crossbar board causes all traffic in the entire system to be disabled. These are the following steps required after detection of a failed Crossbar board. 1. Detect failed Crossbar board. For example, CPU is indicated of loss of connectivity from this board to any one of the existing port boards. 2. Immediately disable TDM and non-TDM traffic for all ports in the Schedulers. 3. Disable the AIB links to/from the Port board. 4. Replace failed Crossbar board. 5. Bring up new board (see "Crossbar Board being added:" on page 90) 6. Enable TDM and best-effort traffic for all ports in the Scheduler. 7. Traffic can now flow. The steps needed to cope with a failed Crossbar board on a redundant Crossbar configuration are described here. A failed Crossbar board does not require traffic to be disabled. The redundant Crossbar board carries traffic while the failed Crossbar board is replaced. These are the necessary steps to follow. 1. Detect failed Crossbar board. For example, CPU is indicated of loss of connectivity from this board to any one of the existing port boards. 2. Disable the AIB links to/from the Port board. 3. Replace failed Crossbar board. 4. Bring up new board (see "Crossbar Board being added:" on page 90) 5. Crossbars are now in fault tolerant mode.
100
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.9.6.3
Scheduler Board
In the non-redundant Scheduler configuration, a failed Scheduler board will prevent all traffic from flowing in the entire system. The following steps are required to cope with this failure. 1. Detect failed Scheduler board. For example, CPU is indicated of loss of connectivity from this board to any one of the existing port boards. 2. Disable TDM and non-TDM traffic to/from ports that still have connectivity. 3. Disable the AIB links to/from the Port board. 4. Replace failed Scheduler board. 5. Bring up new board (see "Scheduler Board being added:" on page 89) 6. Configure this Scheduler to operate in non-redundant Scheduler mode. 7. Enable TDM traffic in Scheduler for connected ports. TDM traffic can now flow. 8. Refresh state information from all ports to this Scheduler. (See Section 1.9.4 "Scheduler Refresh Procedure") 9. Best-effort traffic can now flow. The steps required for a redundant Scheduler configuration are more complicated. The redundant Scheduler can still schedule traffic while the failed Scheduler board is replaced. When the new Scheduler is inserted into the system, it must be resynchronized with the other Scheduler and the Enhanced Port Processors. This will be accomplished by the following steps. 1. Detect failed Scheduler board. For example, CPU is indicated of loss of connectivity from this board to any one of the existing port boards. This failure will cause this Scheduler to be disabled. 2. Disable the AIB links to/from the Port board. 3. Replace failed Scheduler board. 4. Bring up new board (see "Scheduler Board being added:" on page 89) 5. Configure this Scheduler to operate in redundant Scheduler mode. 6. If the Scheduler is generating the TDM sync, then TDM traffic must be stopped by writing a value of 0 to the TDM Sync Period and then writing the actual value to both Schedulers. 7. Enable TDM traffic in new Scheduler for appropriate ports. NOTE: The redundant Scheduler is still scheduling TDM and non-TDM traffic. 8. Refresh state information from all ports to this Scheduler. (See Section 1.9.4 "Scheduler Refresh Procedure") TDM traffic still flows. 9. Best-effort traffic can now flow. Schedulers are now in fault tolerant mode and synchronized.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
101
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.10
ETT1 SIGNALS AND INTERCONNECTIONS
This section describes the different types of signals/interconnects used in a ETT1 switch core. Figure 44 shows the four types of devices required and lists the types of signals present.
Figure 44. ETT1 Signals and Interconnects
C Linecard Line Coder or FO Xceiver C H Dataslice 0
Dataslice 6 O
A
Crossbar Slice Crossbar Slice
A O O H H
A C: LVCMOS @ 150Mb/s H: HSTL @ 200Mb/s (Class 1 Drivers) O: LVCMOS @ 25Mb/s A: STI (PECL-like) @800Mb/s O O O EPP A
FC Crossbar
Scheduler
O Local OOB Bus master
1.10.1
LVCMOS (C and O)
These are standard 2.5V LVCMOS I/Os (3.3V tolerant inputs). The `C' signals provide the interface to the linecoder (if present) such as a Serdes device. These are intra-board signals. The `O' signals are the local OOB bus, and operate using a local 25MHz clock. See Section 1.7 "The Out-of-Band (OOB) Bus Interface" on page 77 for more information.
1.10.2
HSTL (H)
The HSTL signals provide the high-speed intra-board links between the EPP and the Dataslices. These are point-to-point signals. All signals have only one driver. These point-to-point signals carry the data between the first two Dataslice devices and the EPP. These signals are HSTL Class 1 (50 Ohm load).
102
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.10.3
AIB Links (A)
The inter-board device-to-device links are all AIB serial links. These serial links are full duplex with 1.6Gbits/s data in both directions. Each unidirectional link comprises three pairs of differential signals: Even_Data, Odd_Data and Clock. The two data pairs carry link data while the clock provides source-synchronous clock information. The clock is a 400MHz signal. Data is transmitted using both edges of the clock to provide 800Mb/s on each of the two data pairs as shown in Figure 45.
Figure 45. AIB Links
Even_Data+ Even_DataOdd_Data+ Transmitter Odd_DataClock+ Clock1.6 Gb/s Receiver 6 Transmitter Receiver
Even_Data+
Odd_Data+ Clock+
The AIB links are very straightforward to use, and are the same in all four devices. After a link is first enabled it will go through a training sequence with its peer. This is to ensure that the link is operating correctly before data is transmitted. All data carried over AIB links are protected by a CRC field; the status of the link is always visible to the local CPU via registers that can be accessed by the OOB bus. The local CPU must configure all appropriate links before the links can operate. Figure 46 shows the AIB control/status signals that are visible to the CPU.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
103
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 46. AIB Control/Status Signals.
Duplex AIB Link AIB Tx Enable AIB Reset OOB Registers AIB Ready AIB CRC err
Assuming that the link is connected to an AIB peer, then the first step is to enable the transmitter (AIB_Tx_ Enable) and reset the link (AIB_Reset). The driver is inactive until AIB_Tx_Enable is set to 1, as indicated in the Figure 46. The link will start to train as soon as reset is deasserted. Assuming that the link trains correctly then the AIB_Ready signal for this link will be asserted. If the AIB_Ready signal is asserted then the bidirectional link is operational, so both receivers have trained correctly. The link will now be operating in normal mode, transmitting data frames. If a transient error corrupts a frame, causing a CRC mismatch, then the AIB_CRC_err signal will be asserted. If four consecutive frames encounter CRC errors then the receiver will consider the link to have failed. In this case the training sequence will be restarted and AIB_Ready signal will be deasserted. Of course a link might fail, restart and retrain faster than software can respond to an interrupt. In that case the ETT1 devices have separate OOB registers that indicate whether the AIB_Ready signal has transitioned. Thus, software can determine which link is having a problem. A failing link will also cause the AIB_CRC_err bit to be set.
1.10.4
Live Insertion
The AIB drivers should remain disabled until the attached receiver has reached its intended operating voltage. So do not enable an AIB driver unless the peer AIB receiver is inserted in the system and operating (i.e. voltage levels are steady). At reset all AIB drivers will be disabled (tri-stated). If the CPU detects that a board is being withdrawn from the system then it should immediately assert AIB reset for the link that is not being removed and deassert AIB_Tx_Enable for the AIB drivers at both ends of the link.
1.11
SYSTEM LATENCIES
The ETT1 Chip Set operates in a cell-synchronous mode. Therefore all event latencies described here are defined in terms of ETT1 cell times (40ns). Further, all events are defined relative to the Dataslice to linecard interface (Serdes interface).
104
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.11.1
LCS Request to Grant Minimum Latency
This is the minimum latency from when the EPP receives a request from the linecard (1 in Figure 47.) to when the EPP sends an LCS grant (2). These are relative to the Dataslice interface as shown, not the EPP. This minimum latency is 16 cell times.
Figure 47. Request to Grant latency
1 2
Input Data Slice Crossbar Output Data Slice
Input EPP
Flow Control Crossbar
Output EPP
Scheduler
1.11.2
LCS Grant to Linecard Cell Latency
This is the maximum latency from when the iEPP issues an LCS grant (1 in Figure 48.) to when the iEPP expects to receive the cell body associated with the grant (2). These are relative to the Dataslice interface as shown, not the EPP. The maximum latency is 42 cell times minus the programmed value of the "DSiFIFO Token Mechanism Delay" field of the EPP EIDMA register (0009Ch). This is explained in more detail in C.2.2.1 , "Synchronization," on page 333.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
105
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 48. ETT1 Event Latencies
2 1
Input Data Slice Crossbar Output Data Slice
Input EPP
Flow Control Crossbar
Output EPP
Scheduler
1.11.3
Cell latency Through an Empty System
This is the minimum cell latency that a cell body can experience as it passes from the input Dataslice (1 in Figure 49.) to the output Dataslice (2). The minimum cell latency is 29 cell times.
106
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 49. Cell Latency
1
Input Data Slice Crossbar Output Data Slice
2
Input EPP
Flow Control Crossbar
Output EPP
Scheduler
1.11.4
LCS Hole Request to Hole Grant Latency
This is the maximum latency from when a hole request arrives at the Dataslice (1 in Figure 47.) to when the hole request is granted (2). The egress cell does not contain a valid cell at the requested priority. This maximum latency for ETT1 is 64 cell times.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
107
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.12
ETT1 DIAGNOSTICS
An ETT1-based switch is a highly sophisticated system which is typically required to demonstrate very high levels of uptime. In order to achieve this uptime it is important to be able to: * * * Ensure that a subsystem is operating correctly before it is brought online. Identify subsystems that have failed as quickly as possible. Diagnose failures to determine the specific entity that should be replaced.
This document describes a number of diagnostics that customers might find useful in addressing these needs. The following terminology is used to describe a subsystem within a switch: offline: the subsystem is not interacting with the rest of the system except via the OOB bus. AIB links may be taken up and down but will not cause changes to other subsystems. online, inactive:the subsystem may be receiving signals from the system, and may be sending cells to itself, thus contending for fabric bandwidth. It only sends/receives diagnostic cells, not customer traffic. online, active:the subsystem is participating in the normal operation of the switch, sending and receiving customer traffic and maintenance/diagnostic cells.
1.12.1
Device Tests
All of the ETT1 devices have a number of registers. The local ETT1 CPU can read from and write to many of these registers. Some care must be taken when modifying the contents of registers as they may have side effects that will affect the operation or cell flow of system components that are already online and active. In general, configuration registers should not be modified once a subsystem is online and active. The CPU can read most registers at any time without effect, however some registers, particularly interrupt registers and statistics counters, are cleared when they are read. Some of the ETT1 devices contain memory structures (RAMs). Many of these RAMs can be written to and read from by the local CPU. Since the internal BIST mechanism is not available to the CPU, it is recommended that a customer verify the correct operation of such memory structures by a soak test in which the CPU writes and reads to all locations. Such a test should perform the widely known test patterns such as walking 1's, 0's, checkerboard, etc. Of course these tests are destructive to the current contents and can only be performed on RAMs that are not being used. In practice very few of the RAMs can be modified once the devices are active. The RAMs can be read non-destructively, and a great deal of internal system information is available this way; however unless the exact state of the system is known at the instance the RAM is read, then it is difficult to know if the returned value is correct. Obviously, RAMs that implement TDM and multicast tables are designed to be modified with due care during normal operation.
108
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
1.12.2
AIB Link Tests
All of the ETT1 devices use the high-speed AIB links for inter-device communication. The AIBs are used exclusively for inter-device links between PCBs. The AIB links associated with a subsystem can be tested while the subsystem is offline. The following actions describe the procedure that should be used to test an AIB link attached to an offline subsystem (one end of the link may be attached to an online subsystem). 1. If the other end of the AIB is attached to an online (active) subsystem then ensure that the AIB is ignored by that subsystem, as follows: * * * * Dataslice: this (test) AIB should be the secondary link. EPP: this (test) AIB should be the secondary link. Scheduler: disable the port associated with this (test) AIB. (Use the Enable Port register). Xbar: no action needed.
2. Assert the reset control to both ends of the test AIB. 3. Enable the transmitters at both ends of the AIB. 4. De-assert the reset control to both ends of the test AIB. 5. The link should now come up -- the Ready bits associated with the link should be `1'. Interrupts will also be generated if the mask bits are set appropriately. 6. Disable the transmitter at one end of the AIB. 7. The far end of the AIB should indicate CRC errors and that the link is not ready. (A few CRC errors are generated before the link is considered down.) 8. Repeat steps 2 - 6 as needed to ensure that the link is OK. Also, leave the link up for a soak test and ensure that no CRC errors are received.
1.12.3
Linecard to ETT1 Link Diagnostics
When a new portcard in installed in the ETT1 Fabric it will be necessary to ensure that the portcard is operating correctly before bring it online. This section describes a number of tests that can be performed to verify the linecard-to-ETT1 side of the card. Figure 50 shows an example architecture of the portcard and its connection to the linecard on the left and the ETT1 CPU on the right. The Dataslice is shown with two sets of queues.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
109
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 50. Illustrating the Flow of Cells from the Linecard CPU to the ETT1 CPU
Portcard Linecard Dataslice (6 of) Serdes
CPU
Fiber Link
EPP
OOB Satellite
OOB Controller
ETT1 CPU
The shallow queues on the left are simply rate-matching FIFOs of 8 cells depth, used to compensate for asynchrony between the ETT1 core 200MHz clock and the 150MHz Serdes clock. The deeper queues on the right represent the virtual output queues and virtual input queues, where cells are stored pending being forwarded to their egress port or on to the linecard. We assume the presence of a linecard CPU which can generate and receive Control cells to/from the ETT1 portcard
1.12.3.1
Verifying the Fiber Link
In order to verify that the optical fiber link between linecard and portcard is operational, the portcard can set its local Serdes devices to operate in remote loopback mode. In addition the Dataslice devices can operate in a remote loopback mode in which cells received at the ingress are looped back directly to the egress rate matching FIFOs. Using either point of loopback the linecard can send cells to the portcard and have them looped back directly so that the linecard can check the cells itself. This can provide a useful post-failure diagnostic to determine if a fault is in the link or within the portcard or linecard. Figure 51 shows the loopback path.
110
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 51. Loopback Path
Portcard Linecard Dataslice (6 of) Serdes
CPU
Fiber Link
EPP
OOB Satellite
OOB Controller
ETT1 CPU
The loopback function is controlled from the portcard, and so the linecard must have some form of out-of-band communication in order to set either the Serdes or Dataslice into loopback mode. NOTE: While the Dataslice is in loopback, the ingress cells are still received and passed to the EPP. The EPP should be disabled to prevent it from attempting to forward the incoming cells while the portcard is in loopback mode. Also, the EPP must be programmed to send only idle cells. (Set the EIDLCT register to 0x0.) NOTE: If a Mux device is used then the mux bits in the LCS header should be set correctly so as to be returned to the correct OC-48c port. These mux bits will not be modified when the portcard is in loopback mode, and thus must be set correctly for both ingress and egress directions. Some Serdes parts can be set to loopback at their serial outputs - so that the egress cells can be looped back to the ingress direction of the portcard. A portcard could use this feature to test the Serdes logic without needing a linecard to be attached.
1.12.3.2
Establishing Linecard to ETT1 Communication.
The LCS protocol provides support for Control packets which can be exchanged between the linecard CPU and the ETT1 CPU. Figure 52 shows the flow of cells between CPUs.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
111
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 52. Control Packet Exchange Between Linecard CPU and ETT1 CPU
Portcard Linecard Dataslice (6 of) Serdes
CPU
Fiber Link
PP
OOB Satellite
OOB Controller
TT1 CPU
The linecard can send a Control packet to the portcard (once the fiber link is operational). This Control packet is stored in a special input queue that can store up to eight of these packets. Once the packet is stored the EPP can issue an interrupt to the ETT1 CPU indicating the arrival of a Control packet. The ETT1 CPU can then read the Control packet directly from the Dataslice memory. NOTE: The ingress Control packet queue is eight cells deep and there is no flow control imposed on the Control packets sent from the linecard - if the linecard sends control packets faster than the ETT1 CPU can read them then Control packets will be lost. Some form of reliable flow control should be handled between the two CPUs. In the egress direction, the ETT1 CPU can create a Control packet within the Dataslice memory. Once created the CPU issues a write command to the EPP and the cell will be transmitted to the linecard. This mechanism allows the ETT1 CPU to communicate directly with the linecard. This also tests the fiber link, although at a limited cell rate.
1.12.3.3
Testing Error Detection Logic
The ETT1 CPU can send any arbitrary cell to the linecard CPU. The ETT1 CPU simply writes the entire cell into the Dataslice memory, then instructs the EPP to send it by writing to the EPP's "Output UC Queue
112
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Information Memory". Therefore, the ETT1 CPU can create a cell that has an incorrect CRC value. This can be used to test the CRC detection logic at the linecard. Also, the Dataslice uses internal tables to encode and decode the symbols sent to/received from the Serdes. Thus it is possible to configure these tables so that they will produce erroneous codes. This can be used to check that the 8b/10b decode is operating correctly.
1.12.4
ETT1 Internal Datapath Tests
The previous section describes tests that can verify the integrity of the link between the linecard and the ETT1 fabric. This section describes a test that can verify the internal datapaths. Figure 53 shows the datapaths that will be tested.
Figure 53. Testing the ETT1 Internal Datapaths
Portcard
Dataslice (6 of) Serdes Xbar (6 of)
Fiber Link
EPP
Scheduler
OOB Satellite
OOB Controller
ETT1 CPU
The objective is to create a packet in the Dataslice and have it traverse the usual internal ETT1 sequence of Scheduler-request, Scheduler-grant, forward-cell. The steps are as follows: 1. The CPU writes to the Dataslices, creating a cell in one of the virtual output queues corresponding to itself -- i.e. it is going to send a cell to itself.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
113
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
2. Turn off the EPP Output Scheduler for cells coming from itself. This will ensure that the cell will stay in the virtual input queue in the egress buffers of the Dataslice. 3. Bring up all AIB links and enable non-tdm traffic in the Scheduler for this port. Enable the EPP and take it out of LCS-2 Stop mode. 4. Write to the Outstanding Scheduler Request Counter in the EPP for this VOQ. This will cause a request to be made to the Scheduler which will immediately issue a grant to the EPP. The cell will be forwarded through the crossbar and back to the test port. The cell will then be stored in the egress queue but will not be forwarded to the linecard because the Output Scheduler was disabled in step 2. 5. The CPU can read the cell that is waiting in the egress queue and check that it passed through OK. 6. Repeat as needed. A number of variations can be done on this basic theme: * Because the Output Scheduler in each EPP can be stopped on a per source port basis, then in step 1 the CPU can create a cell going to any egress port, not just to itself. This is possible even if the egress port is currently online with real traffic. Thus it is possible to verify the datapath from the new port to all other ports without interfering with existing flows. Do not turn off the Output Scheduler in step 2 (or unfreeze it after Step 6). The linecard can then verify that the cell is OK, thus testing more of the system. Send enough cells so that the egress queue fills up and backpressures the input. So after sending 64 cells (OC-192 configuration) the egress queue should be full and thus prevent any more cells being sent to this queue.
*
*
114
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
2
Dataslice
This chapter contains information on the Dataslice device, part number PM9313-HC, available from PMC-Sierra, Inc. The Dataslice contains queue storage for all cells that have arrived at the input port from the linecard and all cells that have passed through the Crossbar to the output port before departing to the linecard. The Dataslice interfaces to the linecards with 8B/10B encoded physical links, operating at 1.5 Gbit/s. Each Dataslice device packages two independent logical Dataslice units. The ingress and egress datapath is striped between Dataslices such that the first six bytes of the LCS cell are sent through logical Dataslice 0; the next six bytes through logical Dataslice 1, etc. Six Dataslice devices are required on each port card in order to process 72-byte LCS cells. An alternative configuration uses seven Dataslice devices to process 84-byte LCS cells, carrying a 76-byte payload.
2.1
DATASLICE BLOCKS
The following discussion assumes 12 logical Dataslices per port. (6 devices). There are 12 logical Dataslice units, packaged in six ICs, on a 72-byte LCS cell port card. Each logical Dataslice unit has a full-duplex, 10-bit wide, parallel bus connection to a Serdes device on the linecard interface, two bidirectional AIB connections to two Crossbars, two 8-bit busses for data from the ingress and egress paths to the Enhanced Port Processor, one bus for processed ingress data from the EPP, and two unidirectional 8-bit busses for ingress and egress control information from the EPP. Figure 54 shows how cells flow through the Dataslice.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
115
Released Data Sheet
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
PMC-2000164
ISSUE 3
ENHANCED TT1TM SWITCH FABRIC
Figure 54. Dataslice Data Flow
8B/10B oFIFO oQM oBypass AIB
Line Card Interface
Encoder CR 8B/10B iBypass AIB
iFIFO
iQM
Decoder core clocks
OOB Interface Control/Status Registers
JTAG
PLL
ref_clk(n) plllock
Request/Data from iDS d2p_d[1:0]_id
oob_devsel[3:0]
The iFIFO is an eight-cell-deep and the oFIFO is a four-cell-deep FIFO that perform clock decoupling functions between the port card and the linecard and provide buffering between the queue memory and the outside world. The iBypass and oBypass registers are temporary storage areas; eight-cell-deep FIFOs that provide the cell bypass function of the Dataslice. When a cell arrives from the Linecard, nine of the 12 Dataslices store the cell temporarily in their iBypass register. The three remaining Dataslices send their portion of the cell to the EPP. The EPP instructs the Dataslices to transfer the cell from the iBypass register to a specific input queue. The same general flow occurs for the oBypass, except that all 12 Dataslices use the oBypass. There is a constant, programmable depth for each of the bypass registers. This depth is set during initialization through the Out-Of-Band (OOB) bus by the CPU. The iQM and oQM are the on-device queue memories used for storing cells in the input queues and output queues, respectively. The iQM and oQM queue memories are each 8,704 cells deep and 48 bits wide.
116
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Grant/Data to iDS p2d_d[1:0]_id
Data from oDS d2p_d[1:0]_od
Control to oDS p2d_oc
Control to iDS p2d_ic
oob_ad[7:0] oob_valid_L
oob_wait_L
oob_int_lo
oob_int_hi
soc_in(n) soc_out
oob_clk
jtag_tdi jtag_tms jtag_tck jtag_trst_L jtag_tdo
Crossbar Interface
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
2.1.1
OOB Interface and Control/Status Registers
All of the devices have an OOB interface. This interface allows a single local CPU to control and monitor all of the devices within a core fabric. Internally, each device provides registers that can be mapped into the CPU's address space. These registers are described in more detail in Section 2.3 "Dataslice Registers".
2.1.2
2.1.2.1
Dataslice Cell Flow
Input Side Data Flow
On the input side, LCS cells are sent to the Dataslice from the Serdes interface and enter the iFIFO. The header and first four bytes of data enter DSs 0 and 1, the next six bytes of data enter DS2, and the last six bytes of the data enter DS11, as illustrated in Figure 55. Every cell time, one cell is removed from the iFIFO and sent to the iBypass. If the Dataslice is DS0, DS1, or DS2, the cell is copied and sent through the Request/Data from iDS, on d2p_d[1:0]_id, to the EPP. If the iFIFO is empty, the DSs write idle (and non valid) frames to the EPP and iBypass.
Figure 55. Slicing of a Data Cell into Logical Dataslices
Incoming Cell Bytes 12-17 Bytes 18-23 Bytes 24-29 Bytes 30-35 Bytes 36-41 Bytes 42-47 Bytes 48-53 Bytes 54-59 Bytes 60-65 Bytes 66-71 DS9 Bytes 6-11 Bytes 0-5 DS0 DS1 DS2
DS3
DS4
DS5
DS6
DS7
DS8
DS10 DS11
Byte 0-5 Byte 12-17 Byte 24-29 Byte 36-41 Byte 48-53 Byte 60-65 Byte 6-11 Byte 18-23 Byte 30-35 Byte 42-47 Byte 54-59 Byte 66-71
NOTE: For more information on Dataslice synchronization, see Section 2.2.1 "Synchronization" on page 333. The Control to iDS, p2d_ic, provides the write address and write request. If the write request is valid, the cell is written to the iQM location specified by the write address. For DS0, DS1, and DS2, the data from Grant/data to iDS, p2d_d[1:0]_id, is written to memory. For DS3 through DS11, the head cell from the iBypass is written to memory. If the write request is not valid, nothing is written to memory. The cell being transferred out of the head of the iBypass registers is dropped.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
117
Released Data Sheet
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
PMC-2000164
ISSUE 3
ENHANCED TT1TM SWITCH FABRIC
Regardless of the Control to iDS information (on p2d_ic), the Credit Information (byte 0 through bit 5 of byte 4) of the cell header being written on Grant/data to iDS, is saved in a CR (credit register); it is used to indicate backpressure information back to the linecard. The CRC contained in the LCS header of the frame coming from the Grant/Data to iDS is computed only over the Tag portion of the LCS header. Another CRC in the frame that is computed only over the Credit portion of the LCS header is sent to DS0. This credit CRC is stored in the CR with the credits. The Control to iDS frame contains a read request and read address. If the read request is asserted, the iQM location specified by the read address is read and the cell is sent through the Crossbar AIB Interface. The routing tag carried in the Control to iDS link is placed at the head of the cell as it is sent to the Crossbar. If the "SendZero" bit is set, the DS sends an all-zero frame into the Crossbar instead of reading the cell from the queue memory.
2.1.2.2
Output Side Data Flow
Data cells entering the oDS devices from the Crossbar AIB Interface are placed into the oBypass. For DS0, DS1, and DS2, the cells are sent to the EPPs via the Data from oDS busses, d2p_d[1:0]_id. The VLD bit sent to the EPPs is the same as the VLD bit received from the Crossbars. The EPP, having decided to store this new cell, sends a control frame over the Control to oDS bus, p2d_ic. This frame contains a write address and write request. If the write request is asserted, the cell at the head of the oBypass is placed into the oQM location designated by the write address. If the write request is not asserted, the cell at the head of the oBypass is removed from the FIFO and discarded. The Control to oDS frame also contains a read request and read address. If the read request is asserted, then the cell in the oQM location specified by the read address is removed from the memory and placed in the oFIFO. If the "SendZero" bit is set, the DS inserts an all-zero frame into the oFIFO instead of reading a cell from queue memory. As the cell is inserted into the oFIFO, bytes 0 through the first part of byte 4 (the Credit Information fields of the LCS header) are overwritten by the contents of the CR (Credit Register) by DS0, and the LCS CRC is computed as the XOR of the tag CRC contained in the cell and the Credit CRC contained in the CR. If the Serdes clock rate is slightly less than the core clock rate, this oFIFO occasionally fills up. The EPP can be programmed to insert one "idle" cell periodically into the stream entering the oFIFO to avoid filling the oFIFO. The EPP does not assert either a read request or a "send zero" command to the oDS for these "idle" cells, nor does the oDS insert any cell into the oFIFO. The frequency of these idle cells is programmable and determined by the relative clock frequencies of the switch core and the linecard.
2.2
8B/10B INTERFACE
The Dataslice connects to the outside world via a 10-bit wide transmit and receive interface. In the ingress direction, an 8B/10B decoder lookup table maps received 10-bit characters into LCS data bytes. In the egress direction, an 8B/10B encoder lookup table maps LCS data bytes into 10-bit characters to be transmitted. The 8B/10B encoder and decoder lookup tables can be programmed to use any arbitrary 8B/10B code.
118
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
2.3
DATASLICE REGISTERS
The Dataslice device select low-order bits are hard wired on the board by three pins (oobdev_sel[2:0]) on the Dataslice package. Since there are two logical Dataslices within each package, the fourth implied bit is used to select which of the two logical Dataslices is addressed.
2.3.1
Dataslice Summary
The following table is a summary of information for all registers in the Dataslice. See the following Descriptions section for more information on individual registers. Read and Clear means that reading the register causes it to be cleared (reset to zero)
Table 21. Dataslice Register Summary
Address 00000h 00004h 00008h 0000Ch 00010h 00030h 00034h 0003Ch 00080h 00084h 00088h 0008Ch 00090h 00094h 00098h 0009Ch 000A0h
Symbol DSTS DRS DIRLMSK DIRHMSK DIR DAIBRS DAIBRDY DAIBEN DPRIXB DINBY DOUTBY DCODEN DDECEN DLSRRS DLSRACT DPINRS DPINDET Status Reset
Register
Access Read Only Read/Write Read/Write Read/Write Read and Clear Read/Write Read Only Read/Write Read/Write Read/Write Read/Write Read/Write Read/Write Read/Write Read Only Read/Write Read Only
Default Value 10000000h 00000004h 00000000h 00000000h 00000000h 00000003h 00000003h 00000000h 00000000h 00000002h 00000002h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h
Low Priority Mask High Priority Mask Interrupt Register AIB Reset AIB Ready AIB Tx Enable Primary Crossbar Select Input Bypass Shift Register Depth Output Bypass Shift Register Depth 8B/10B Encoder Enable 8B/10B Decoder Enable VCSEL Laser Reset (Active-Low) VCSEL Laser Active Status PIN Diode Reset (Active-Low) PIN Diode Signal Detect
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
119
Released Data Sheet
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
PMC-2000164
ISSUE 3
ENHANCED TT1TM SWITCH FABRIC
Table 21.
Dataslice Register Summary (Continued)
Address 000A4h 000A8h 000ACh 000B0h 00100h 40000-5FFFCh 60000- 7FFFCh 80000-807FCh 81000-81FFCh
Symbol DXCVDET DXCVLB DFORDY DINTLB DPLL DIQM DOQM D8BCOD D8BDEC
Register Transceiver Signal Detect Transceiver Loopback Mode Fiber Optics Link Ready Dataslice Loopback Mode PLL Control/Status Input Queue Memory (IQM) Output Queue Memory (OQM) 8B/10B Encoder Lookup Table 8B/10B Decoder Lookup Table
Access Read Only Read/Write Read Only Read/Write Read/Write Read/Write Read/Write Read/Write Read/Write
Default Value 00000000h 00000000h 00000000h 00000000h 0001447Ch Unknown Unknown Unknown Unknown
120
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
2.3.2
2.3.2.1 Symbol:
Dataslice Register Descriptions
Status DSTS
Address Offset: 00000h Default Value: 10000000h Access: Read Only Status register. Bits 31:28 27:24 23:2 1 0 Device Revision Number. Reserved. High Priority Interrupt. 1 = an outstanding high priority interrupt. One of the bits in the Interrupt Register is set, and is enabled via its corresponding high priority mask. Low Priority Interrupt. 1 = an outstanding low priority interrupt. One of the bits in the Interrupt Register is set, and is enabled via its corresponding low priority mask. Description Device ID Number. Identifies the specific device.
2.3.2.2 Symbol:
Reset DRS
Address Offset: 00004h Default Value: 00000000h Access: Read/Write Reset register. Bits 31-1 0 Reserved. Reset. Writing a 1 to this location will reset the entire device. It is equivalent to a hardware reset. This register is cleared automatically when the device is reset. Writing a 0 is not necessary. Soft reset takes 1mS to complete. Description
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
121
Released Data Sheet
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
PMC-2000164
ISSUE 3
ENHANCED TT1TM SWITCH FABRIC
2.3.2.3 Symbol:
Low Priority Mask DIRLMSK
Address Offset: 00008h Default Value: 00000000h Access: Read/Write Interrupt Mask for interrupts. Bits 31:24 23:0 Reserved. Low Priority Mask. Mask bits for low priority interrupts. Each mask bit is set to 1 to enable a low priority interrupt when the corresponding bit in the Interrupt Register is 1. Description
2.3.2.4 Symbol:
High Priority Mask DIRHMSK
Address Offset: 0000Ch Default Value: 00000000h Access: Read/Write Interrupt Mask for interrupts. Bits 31:24 23:0 Reserved. High Priority Mask. Mask bits for high priority interrupts. Each mask bit is set to 1 to enable a high priority interrupt when the corresponding bit in the General Interrupt Register is 1. Description
122
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
2.3.2.5 Symbol:
Interrupt Register DIR
Address Offset: 00010h Default Value: 00000000h Access: Read and Clear Interrupt Register. Bits 31:24 23 22 Reserved. 8B/10B Decoder Error. This is set if a 10B character received from the fiber optics interface is not a valid 8B/10B code as defined in the 8B/10B decoder lookup table. Output Fifo Overflow. This is set if the EPP attempts to push a cell into the output fifo when it is full. This output fifo can hold up to 4 cells. This is likely to be caused by incorrect programming of the "1-in-N" counter on the EPP. Input Fifo Overflow. This is set if the fiber optics receiver attempts to push a cell into the input fifo when it is full. The input fifo can hold up to 8 cells. This is likely to be caused by a malfunction in the "1-in-N" counter on the ingress linecard. VCSEL Laser Active Up. This is set if the VCSEL laser array raises its normal operation status. VCSEL Laser Active Down. This is set if the VCSEL laser array drops its normal operation status. PIN Diode Signal Detect Up. This is set if the PIN diode receiver raises the detection of a light stream from the fiber optical link. PIN Diode Signal Detect Down. This is set if the PIN diode receiver drops the detection of a light stream from the fiber optical link. Transceiver Signal Detect Up. This is set if the transceiver raises the detection of a bit stream from the fiber optics PIN diode receiver. Transceiver Signal Detect Down. This is set if the transceiver drops the detection of a bit stream from the fiber optics PIN diode receiver. Transceiver Comma Detect. This is set if the transceiver indicates the detection of a comma character "0011111xxx". 8B/10B Decoder Comma Detect. This is set if the fiber optics receiver looks up a 10B control character that maps to "0100000000" in the 8B/10B decoder lookup table. Fiber Optics Egress Ready Up. This is set if the fiber optics receiver on the egress linecard raises its ready status as a consequence of synchronization being locked on the 8B/10B control sequences. Fiber Optics Egress Ready Down. This is set if the fiber optics receiver on the egress linecard drops its ready status as a consequence of synchronization being lost on the 8B/10B control sequences. Description
21 20 19 18 17 16 15 14 13 12
11
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
123
Released Data Sheet
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
PMC-2000164
ISSUE 3
ENHANCED TT1TM SWITCH FABRIC
Bits 10 9 8 7 6 5 4 3 2 1 0
Description (Continued) Fiber Optics Ingress Ready Up. This is set if the fiber optics receiver raises its ready status as a consequence of synchronization being locked on the 8B/10B control sequences. Fiber Optics Ingress Ready Down. This is set if the fiber optics receiver drops its ready status as a consequence of synchronization being lost on the 8B/10B control sequences. Crossbar 1 Ready Up. This is set if the Crossbar 1 AIB link raises its ready status. Crossbar 0 Ready Up. This is set if the Crossbar 0 AIB link raises its ready status. Crossbar 1 Ready Down. This is set if the Crossbar 1 AIB link drops its ready status. Crossbar 0 Ready Down. This is set if the Crossbar 0 AIB link drops its ready status. Crossbar 0 and Crossbar 1 Ready Down Together. This is set if both Crossbar 0 and Crossbar 1 AIB links drop their ready status simultaneously. Crossbar 1 CRC Error. This is set if a CRC error is flagged by the Crossbar 1 link receiver. Crossbar 0 CRC Error. This is set if a CRC error is flagged by the Crossbar 0 link receiver. Crossbar 0 and Crossbar 1 CRC Errors Together. This is set if CRC errors are simultaneously flagged by both Crossbar 0 and Crossbar 1 AIB link receivers. Crossbar 0 and Crossbar 1 Mismatch. This is set if both Crossbar 0 and Crossbar 1 AIB links are ready, there is no CRC error, and the data received from Crossbar 0 does not match that received from Crossbar 1.
2.3.2.6 Symbol:
AIB Reset DAIBRS
Address Offset: 00030h Default Value: 00000003h Access: Read/Write Used to assert reset to the corresponding Crossbar AIB link. Bit 0 is for the Crossbar 0 AIB link, and bit 1 is for the Crossbar 1 AIB link. If reset is asserted, the link will not operate or transition to ready. Reset is asserted to 1 on power-up reset or if the reset bit in the control register is asserted. Bits 31:2 1 0 Reserved. Crossbar 1 AIB Link Reset Crossbar 0 AIB Link Reset Description
124
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
2.3.2.7 Symbol:
AIB Ready DAIBRDY
Address Offset: 00034h Default Value: 00000003h Access: Read Only Indicates if the corresponding Crossbar AIB link is ready. Bits 31:2 1 0 Reserved. Crossbar 1 AIB link Crossbar 0 AIB link NOTE: Default value assumes that Primary and Secondary Crossbars are installed. Description
2.3.2.8 Symbol:
AIB Tx Enable DAIBEN
Address Offset: 0003Ch Default Value: 00000000h Access: Read/Write Used to enable the transmitter of the corresponding Crossbar AIB link. If the corresponding Crossbar is not physically present in the system, the bit for the Crossbar should be set to zero to reduce electrical noise and to prevent electrical glitches when the Crossbar board is inserted or removed. Bits 31:2 1 0 Crossbar 1 AIB Link Enable Transmitter Crossbar 0 AIB Link Enable Transmitter Description
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
125
Released Data Sheet
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
PMC-2000164
ISSUE 3
ENHANCED TT1TM SWITCH FABRIC
2.3.2.9 Symbol:
Primary Crossbar Select DPRIXB
Address Offset: 00080h Default Value: 00000000h Access: Read/Write Selects Crossbar 0 or Crossbar 1 as the primary Crossbar for fault tolerance. Bits 31:1 0 Reserved. Primary Crossbar Select. A value of 0 selects Crossbar 0. A value of 1 selects Crossbar 1. Description
2.3.2.10 Symbol:
Input Bypass Shift Register Depth DINBY
Address Offset: 00084h Default Value: 00000002h Access: Read/Write Programs the depth of the Input Bypass Shift Register between 0 and 7 cells. This register defaults to a depth of 2 at power-up and reset time and should be changed to a value of 3 for use in a ETT1 system.. Bits 31:3 2:0 Reserved. Depth Range. Values from 0 to 7. Description
126
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
2.3.2.11 Symbol:
Output Bypass Shift Register Depth DOUTBY
Address Offset: 00088h Default Value: 00000002h Access: Read/Write Programs the depth of the Output Bypass Shift Register between 0 and 7 cells. This register defaults to a depth of 2 at power-up and reset time and should be changed to a value of 3 for use in a ETT1 system. Bits 31:3 2:0 Reserved. Depth Range. Values from 0 to 7. Description
2.3.2.12 Symbol:
8B/10B Encoder Enable DCODEN
Address Offset: 0008Ch Default Value: 00000000h Access: Read/Write Enables the 8B/10B Encoder Bits 31:1 0 Reserved. 8B/10B Encoder Enable. This bit enables the 8B/10B encoder logic for the fiber optics transmitter. If the 8B/10B encoder is disabled, no cells are popped from the output fifo. It is generally a good idea to program the 8B/10B encoder lookup table before the 8B/10B encoder is enabled. Description
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
127
Released Data Sheet
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
PMC-2000164
ISSUE 3
ENHANCED TT1TM SWITCH FABRIC
2.3.2.13 Symbol:
8B/10B Decoder Enable DDECEN
Address Offset: 00090h Default Value: 00000000h Access: Read/Write Enables the 8B/10B Decoder. Bits 31:1 Reserved. 8B/10B Decoder Enable. This bit enables the 8B/10B decoder logic for the fiber optics receiver. If the 8B/10B decoder is disabled, no synchronization is locked onto and therefore the fiber optics ready status is not asserted, and no tokens/cells are pushed into the token/input fifos. It is generally a good idea to program the 8B/10B decoder lookup table before the 8B/10B decoder is enabled. Description
0
2.3.2.14 Symbol:
VCSEL Laser Reset (Active-Low) DLSRRS
Address Offset: 00094h Default Value: 00000000h Access: Read/Write Drive the active-low reset line on the Siemens PAROLI VCSEL laser array. NOTE: This is a general output pin and may be used for other purposes. Bits 31-1 0 Reserved. VCSEL Laser Reset (Active-Low). This bit can be used to drive the active-low reset line on the VCSEL laser array. It defaults to a low level at power-up and reset time which causes the laser array to be switched off. Description
128
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
2.3.2.15 Symbol:
VCSEL Laser Active Status DLSRACT
Address Offset: 00098h Default Value: 00000000h Access: Read Only Indicates the status of the Siemens PAROLI laser array. NOTE: This is a general input pin and may be used for other purposes. Bits 31:1 0 Reserved. VCSEL Laser Active Status. This bit indicates whether the VCSEL laser array is in normal operation (1) or there is a laser fault (0) or the reset line is asserted active-low (0). Description
2.3.2.16 Symbol:
PIN Diode Reset (Active-Low) DPINRS
Address Offset: 0009Ch Default Value: 00000000h Access: Read/Write Drives the active-low reset line on the Siemens PAROLI PIN diode array. NOTE: This is a general output pin and may be used for other purposes. Bits 31:1 0 Reserved. PIN Diode Reset (Active-Low). This bit can be used to drive the active-low reset line on the PIN diode array. It defaults to a low level at power-up and reset time which causes the PIN diode array outputs to be driven low. Description
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
129
Released Data Sheet
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
PMC-2000164
ISSUE 3
ENHANCED TT1TM SWITCH FABRIC
2.3.2.17 Symbol:
PIN Diode Signal Detect DPINDET
Address Offset: 000A0h Default Value: 00000000h Access: Read Only Indicates whether the Siemens PAROLI PIN diode array detects a signal of sufficient AC power on the fiber optical link. NOTE: This is a general input pin and may be used for other purposes. Bits 31:1 0 Reserved. PIN Diode Signal Detect. This bit indicates whether the PIN diode array detects a signal of sufficient AC power on the fiber optical link. Description
2.3.2.18 Symbol:
Transceiver Signal Detect DXCVDET
Address Offset: 000A4h Default Value: 00000000h Access: Read Only Indicates whether the transceiver detects a bit stream of sufficient AC power from the PIN diode receiver. NOTE: This is a general input pin and may be used for other purposes. Bits 31:1 0 Reserved. Transceiver Signal Detect. Description
130
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
2.3.2.19 Symbol:
Transceiver Loopback Mode DXCVLB
Address Offset: 000A8h Default Value: 00000000h Access: Read/Write Enables a local loopback mode as provided by the transceiver. NOTE: This is a general output pin and may be used for other purposes. Bits 31:1 0 Reserved. Transceiver Loopback Mode. This bit can be used to enable a local loopback mode as provided by the transceiver. Description
2.3.2.20 Symbol:
Fiber Optics Link Ready DFORDY
Address Offset: 000ACh Default Value: 00000000h Access: Read Only Indicates synchronization and idle-ready state of the fiber optics transmitter. NOTE: This is a general input pin and may be used for other purposes. Bits 31:2 1 Reserved. Egress Link Ready. This bit indicates that the linecard fiber optics transmitter transitioned to the idle-ready 8B/10B control sequence. The linecard should transition from the idle 8B/10B control sequence to the idle-ready 8B/10B control sequence when its fiber optics receiver is synchronized. Ingress Link Ready. This bit indicates that the fiber optics receiver is locked to the synchronization transmitted by the linecard. It causes the fiber optics transmitter to transition from the idle 8B/10B control sequence to the idle-ready 8B/10B control sequence. Description
0
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
131
Released Data Sheet
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
PMC-2000164
ISSUE 3
ENHANCED TT1TM SWITCH FABRIC
2.3.2.21 Symbol:
Dataslice Loopback Mode DINTLB
Address Offset: 000B0h Default Value: 00000000h Access: Read/Write Enables a local loopback mode on the Dataslice. Bits 31:1 0 Reserved. Dataslice Loopback Mode. This bit enables a local loopback mode on the Dataslice that causes data to be looped internally from the input fifo to the output fifo. Description
2.3.2.22 Symbol:
PLL Control/Status DPLL
Address Offset: 00100h Default Value: 0001447Ch Access: Read/Write Controls operation of the internal PLL (Phase Locked Loop). After a power on reset, the PLL itself is held in reset. This is reflected in bit 16 of this register. The local CPU must reset this bit to 0 to enable operation of the device, and thus should write 0000447Ch to this register. Bit 31:17 16 15:0 Description PLL status. These bits reflect internal PLL operation status and should be ignored. Reset PLL. When set to 1 the PLL is held reset. The supplied reference clock will be used as the internal clock. The serial links will not be operational. This bit will be 1 after power-up reset and should be deasserted for normal operation. PLL reset takes 10mS to complete. PLL control.
132
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
2.3.2.23 Symbol:
Input Queue Memory (IQM) DIQM
Address Offset: 40000-5FFFCh Default Value: Unknown Access: Read/Write The actual IQM address range extends from 40000h to 50FFCh. Beyond IQM address 50FFCh, aliasing occurs within the IQM 512x48 RAM bank. If the IQM is accessed at an even 32-bit aligned word address, the data bitmap consists of the following: Bits 31:16 15:0 Reserved. Description
This field contains the 16 MSBs of the addressed IQM 48-bit word.
If the IQM is accessed at an odd 32-bit aligned word address, the data bitmap consists of the following: Bits 31:0 Description This field contains the 32 LSBs of the addressed IQM 48-bit word
2.3.2.24 Symbol:
Output Queue Memory (OQM) DOQM
Address Offset: 60000- 7FFFCh Default Value: Unknown Access: Read/Write The actual OQM address range extends from 60000h to 70FFCh. Beyond IQM address 70FFCh, aliasing occurs within the OQM 512x48 RAM bank. If the OQM is accessed at an even 32-bit aligned word address, the data bitmap consists of the following: Bits 31:16 15:0 Reserved. This field contains the 16 MSBs of the addressed OQM 48-bit word. Description
If the IQM is accessed at an odd 32-bit aligned word address, the data bitmap consists of the following: Bits 31:0 Description This field contains the 32 LSBs of the addressed OQM 48-bit word
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
133
Released Data Sheet
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
PMC-2000164
ISSUE 3
ENHANCED TT1TM SWITCH FABRIC
2.3.2.25 Symbol:
8B/10B Encoder Lookup Table D8BCOD
Address Offset: 80000-807FCh Default Value: Unknown Access: Read/Write The actual 8B/10B encoder lookup table address range extends from 0x80000 to 0x804FC. Beyond 8B/10B encoder lookup table address 0x804FC aliasing will occur within the 8B/10B encoder lookup table 320x26 RAM. The 8B/10B encoder lookup table address space is defined as: * * * 0x80000-0x803FC: Lookup of 8B data bytes into 10B data characters. 0x80400-0x80414: Lookup of idle 8B/10B control sequence (suggested 10B control sequence: 0x80400 = K28.5 and 0x80404-0x80414 = K27.7). 0x80420-0x80434: Lookup of idle-ready 8B/10B control sequence (suggested 10B control sequence: 0x80420 = K28.5 and 0x80424-0x80434 = K29.7).
The 8B/10B encoder lookup table data bitmap consists of the following: Bits 31:26 25:23 22:13 12:10 9:0 Reserved. Weight of the positive RDS 10B character. Positive RDS 10B character. Absolute weight of the negative RDS 10B character. Negative RDS 10B character. NOTE: Bit 0 and bit 13 represent the bit that would be transmitted first for each respective 10B character. Description
134
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
2.3.2.26 Symbol:
8B/10B Decoder Lookup Table D8BDEC
Address Offset: 81000-81FFCh Default Value: Unknown Access: Read/Write The 8B/10B decoder lookup table data bitmap consists of the following: Bits 31:10 9 Reserved. This bit flags an 8B/10B decode error. The entry that the 10B character indexes does not correspond to a valid 8B control or data byte. This bit identifies an 8B/10B control character. If the 10B character indexes an entry that has this control bit set and the data field [7:0] is 0x0, the 10B character is recognized as the comma character for synchronization. If the 10B character indexes an entry that has this control bit set and the data field [7:0] is 0xFF, the 10B character is recognized as an indication that the linecard fiber optics is ready. This data field contains the corresponding 8B value. Description
8
7:0
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
135
Released Data Sheet
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
PMC-2000164
ISSUE 3
ENHANCED TT1TM SWITCH FABRIC
2.4
DATASLICE SIGNAL DESCRIPTIONS
This section describes the Dataslice signals. The following notation is used to describe the signal type: I O B Input pin Output pin Bidirectional Input/Output pin
The signal description also includes the type of buffer used for the particular signal: PECL STI PECL are Pseudo-ECL (positive voltage ECL) compatible signals. STI is a very high-speed asynchronous communications protocol used to connect devices that may be 1 to 15 meters apart. All HSTL specifications are consistent with EIA/JEDEC Standard, EIA/JESD8-6 "High Speed Transceivers Logic (HSTL): A 1.5V Output Buffer Supply Voltage based Interface Standard for Digital Integrated Circuits", dated 8/95. Refer to these specifications for more information. The CMOS Buffers are either 3.3 V compatible or 2.5 V Low Voltage TTL compatible signals, as noted.
Table 22. Dataslice Signal Descriptions
HSTL
CMOS
Name
I/O
Type OOB Interface
Description
oob_ad[7:0] oob_clk oob_devsel0 oob_devsel1 oob_devsel2 oob_valid_L oob_wait_L
B I I I I I O
CMOS CMOS CMOS CMOS CMOS CMOS CMOS
OOB Address bus OOB CLock OOB device select OOB device select OOB device select OOB Control OOB Control
DS0 OOB Interface oob0_int_hi O CMOS OOB Interrupt
136
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 22.
Dataslice Signal Descriptions (Continued)
Name oob0_int_lo
I/O O
Type CMOS OOB Interrupt
Description
DS1 OOB Interface oob1_int_hi oob1_int_lo O O CMOS CMOS OOB Interrupt OOB Interrupt
DS/EPP Interface p2d_ic[7:0] p2d_oc[7:0] Vref I I I HSTL (Class 2) HSTL (Class 2) HSTL EPP to DS ingress control EPP to DS egress control Input Reference Voltage
DS0 DS/EPP Interface d2p_d0_id[7:0] d2p_d0_od[7:0] p2d_d0_id[7:0] O O I HSTL (Class 1) HSTL (Class 1) HSTL (Class 1) DS to EPP ingress data unmodified DS to EPP egress data EPP to DS ingress data modified
DS1 DS/EPP Interface d2p_d1_id[7:0] d2p_d1_od[7:0] p2d_d1_id[7:0] O O I HSTL (Class 1) HSTL (Class 1) HSTL (Class 1) DS to EPP ingress data unmodified DS to EPP egress data EPP to DS ingress data modified
DS0 AIB Interface d2x_d0a_[c, cn, e, en, o, on] d2x_d0b_[c, cn, e, en, o, on] x2d_d0a_[c, cn, e, en, o, on] x2d_d0b_[c, cn, e, en, o, on] O O I I STI STI STI STI DS to Xbar cell data DS to Xbar cell data Xbar to DS cell data Xbar to DS cell data
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
137
Released Data Sheet
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
PMC-2000164
ISSUE 3
ENHANCED TT1TM SWITCH FABRIC
Table 22.
Dataslice Signal Descriptions (Continued)
Name
I/O
Type DS1 AIB Interface
Description
d2x_d1a_[c, cn, e, en, o, on] d2x_d1b_[c, cn, e, en, o, on] x2d_d1a_[c, cn, e, en, o, on] x2d_d1b_[c, cn, e, en, o, on]
O O I I
STI STI STI STI
DS to Xbar cell data DS to Xbar cell data Xbar to DS cell data Xbar to DS cell data
Serdes Interface fotx_clk_out O CMOS Clock output sync'd to fotx_txd
DS0 Serdes Interface forx0_clk forx0_rxd[9:0] fotx0_txd[9:0] pin_diode0_reset_L pin_diode0_sig_det vcsel0_laser_act vcsel0_reset_L xcvr0_com_det xcvr0_loopback xcvr0_sig_det I I O O I I O I O I CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS Serdes to DS 8b/10b receive clock Serdes to DS 8b/10b encoded data DS to Serdes 8b/10b encoded data VCSEL and Serdes control signals VCSEL and Serdes status signals VCSEL and Serdes status signals VCSEL and Serdes control signals VCSEL and Serdes status signals VCSEL and Serdes control signals VCSEL and Serdes status signals
DS1 Serdes Interface forx1_clk forx1_rxd[9:0] fotx1_txd[9:0] pin_diode1_reset_L pin_diode1_sig_det vcsel1_laser_act vcsel1_reset_L I I O O I I O CMOS CMOS CMOS CMOS CMOS CMOS CMOS Serdes to DS 8b/10b receive clock Serdes to DS 8b/10b encoded data DS to Serdes 8b/10b encoded data VCSEL and Serdes control signals VCSEL and Serdes status signals VCSEL and Serdes status signals VCSEL and Serdes control signals
138
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 22.
Dataslice Signal Descriptions (Continued)
Name xcvr1_com_det xcvr1_loopback xcvr1_sig_det
I/O I O I
Type CMOS CMOS CMOS
Description VCSEL and Serdes status signals VCSEL and Serdes control signals VCSEL and Serdes status signals
Power Supply, Clock Source, Reset, and Diagnostics fotx_clk_in plllock pwrup_reset_L VDDA VDD VDDQ GND ref_clk and ref_clkn soc_in and soc_inn I I I O I CMOS CMOS CMOS Isolated Supply Supply Supply Supply PECL PECL Clock input used for fotx_clk_out Test output Synchronous, Active Low Reset VDD (2.5 V) for PLL VDD (2.5 V) VDDQ (1.6 V) GND (0 V) System 200MHz clock (differential) System Start-of-cell (differential)
JTAG Interface jtag_tck jtag_tdi jtag_tdo jtag_tms jtag_trst_L I I O I I CMOS_ 2.5_only CMOS_ 2.5_only CMOS_ 2.5_only CMOS_ 2.5_only CMOS_ 2.5_only 1149.1 JTAG 2.5V ONLY! 1149.1 JTAG 2.5V ONLY! 1149.1 JTAG 2.5V ONLY! 1149.1 JTAG 2.5V ONLY! 1149.1 JTAG 2.5V ONLY!
Dataslice Configuration crdcrcen0 crdcrcen1 I I CMOS CMOS DS0 credit CRC enable tied low on all DS devices DS1 credit CRC enable tied high on physical DS device 0 only
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
139
Released Data Sheet
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
PMC-2000164
ISSUE 3
ENHANCED TT1TM SWITCH FABRIC
Table 22.
Dataslice Signal Descriptions (Continued)
Name crden0 crden1 ibpen0 ibpen1
I/O I I I I
Type CMOS CMOS CMOS CMOS
Description DS0 credit enable tied high on physical DS device 0 only DS1 credit enable tied low on all DS devices DS0 iBypass enable tied low on physical DS device 0 only DS1 iBypass enable tied low on physical DS device 0 only
ASIC Manufacturing Test Interface (see Appendix B.2) ce0_io ce0_scan ce0_tstm3 lssd_ce1_a lssd_ce1_b lssd_ce1_c1 lssd_ce1_c2 lssd_ce1_c3 lssd_scan_in[15:0] lssd_scan_out[15:0] mon[15:0] plltest_in plltest_out test_di1 test_di2 test_lt test_re test_ri I I I I I I I I I O O I O I I I I I CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS Test (GND) Test (GND) Test (GND) Test (VDD) Test (VDD) Test (VDD) Test (VDD) Test (VDD) Test: Scan input (GND) Test: Scan output Test (NC) Test (GND) Test output (NC) Test (VDD) Should be driven to GND during reset. All outputs are tristated when low. Test (VDD) Should be driven to GND during reset. All outputs are tristated when low. Test (VDD) Test (GND) Test (VDD)
140
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
141
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
2.5 2.5.1
PINOUT AND PACKAGE INFORMATION Pinout Tables
Table 23. Dataslice Pinout (left side)
01 A B C D E F G H J K L M N P R T U V W Y AA AB AC AD AE
No Pin d2p_d1_id0 ref_clkn d2p_d1_id4 VDDA d2p_d1_id5 UNUSED p2d_d1_id6 d2p_d1_od1
02
GND crdcrcen1 VDDQ crden1 GND d2p_d1_id2 VDD p2d_d1_id1 GND
03
test_di2 ibpen1 ref_clk oob1_int_lo plltest_in d2p_d1_id1 plltest_out d2p_d1_id7 p2d_d1_id4
04
VDDQ
05
test_di1
06
GND
07
test_ri UNUSED test_lt p2d_ic7 p2d_ic0 p2d_ic5 soc_in
08
VDD oob_valid_L GND p2d_ic2 VDD p2d_ic4 GND p2d_ic6 VDDQ oob_devsel1 GND Vref VDD fotx1_txd9 GND d2x_d1b_c VDDQ x2d_d1b_e GND x2d_d1b_en VDD mon4 GND
09
UNUSED UNUSED UNUSED UNUSED lssd_scan_ out9 UNUSED lssd_scan_ out8 p2d_oc6 p2d_ic1 Vref p2d_ic3 GND fotx1_txd3 VDD mon3 mon2 fotx_clk_in fotx_clk_out lssd_scan_ in9 UNUSED lssd_scan_ in8 UNUSED UNUSED UNUSED UNUSED
10
GND UNUSED VDDQ UNUSED GND UNUSED VDD UNUSED GND p2d_oc7 VDD fotx0_txd7 GND mon6 VDD mon5 GND UNUSED VDD UNUSED GND UNUSED VDDQ UNUSED GND
pwrup_reset_ oob_devsel2 oob_wait_L L GND oob1_int_hi VDDQ p2d_d1_id2 GND test_re UNUSED plllock UNUSED p2d_d1_id3 VDD oob_clk GND soc_inn VDDQ
d2p_d1_id3 d2p_d1_od0 d2p_d1_id6 oob_devsel0 VDD fotx1_txd1 GND UNUSED VDDQ UNUSED GND fotx1_txd5 VDD fotx1_txd7 GND d2x_d1a_e VDDQ xcvr1_sig_ det GND xcvr1_com_ det VDDQ d2p_d1_od3 GND UNUSED Vref p2d_d1_id7 lssd_scan_ out7 UNUSED lssd_scan_ out4 d2x_d1a_o d2x_d1a_en x2d_d1a_en d2x_d1b_on d2x_d1b_cn x2d_d1b_o x2d_d1b_on mon1 lssd_scan_ in10
p2d_d1_id0 d2p_d1_od5 d2p_d1_od2 p2d_d1_id5 UNUSED UNUSED UNUSED d2x_d1a_c x2d_d1a_o forx1_rxd3 VDDQ UNUSED GND UNUSED VDD forx1_rxd1 GND d2p_d1_od7 UNUSED UNUSED UNUSED fotx1_txd8 forx1_rxd2 forx1_rxd4 forx1_rxd5 lssd_scan_ out3 forx1_rxd9 lssd_scan_ out2 pin_diode1_ reset_L lssd_scan_ out1 forx1_clk lssd_scan_ in14
d2p_d1_od6 d2p_d1_od4 fotx1_txd0 lssd_scan_ out6 UNUSED lssd_scan_ out5 fotx1_txd2 fotx1_txd4 fotx1_txd6 d2x_d1a_on x2d_d1a_c x2d_d1a_cn lssd_scan_ in15 d2x_d1b_e lssd_scan_ in12 x2d_d1b_c lssd_scan_ in13 VDDQ UNUSED GND UNUSED VDD forx1_rxd0 GND x2d_d1a_e VDDQ d2x_d1b_o GND xcvr1_ loopback VDD mon0 GND
d2x_d1a_cn x2d_d1a_on UNUSED forx1_rxd6 UNUSED forx1_rxd7 lssd_scan_ out0 UNUSED vcsel1_ laser_act VDDQ forx1_rxd8 GND vcsel1_ reset_L VDD pin_diode1_ sig_det GND
d2x_d1b_en x2d_d1b_cn lssd_scan_ in11 VDD
142
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 24.
Dataslice Pinout (right side)
11
UNUSED UNUSED UNUSED UNUSED lssd_scan_ out10 UNUSED lssd_scan_ out11 p2d_oc5 p2d_oc2 Vref p2d_oc0 VDD fotx0_txd6 GND mon12 mon13 mon10 mon8 lssd_scan_ in6 UNUSED lssd_scan_ in7 UNUSED UNUSED UNUSED UNUSED
12
VDD UNUSED GND p2d_oc4 VDDQ p2d_oc3 GND oob_ad5 VDD UNUSED GND d2p_d0_od1 VDD fotx0_txd1 GND x2d_d0b_c VDD d2x_d0b_o GND mon7 VDDQ mon11 GND d2x_d0b_c VDD
13
lssd_scan_ out13 UNUSED lssd_scan_ out12 p2d_oc1 oob_ad6 oob_ad1 oob_ad2 UNUSED UNUSED
14
GND oob_ad7 VDD oob_ad4 GND UNUSED VDDQ Vref GND
15
lssd_scan_ out15 oob_ad3 lssd_scan_ out14 UNUSED jtag_tdo d2p_d0_id2 p2d_d0_id6 p2d_d0_id1 d2p_d0_od2 Vref fotx0_txd8 ce0_io UNUSED ce0_tstm3 fotx0_txd5 fotx0_txd4 d2x_d0a_on d2x_d0a_e x2d_d0a_c
16
VDDQ oob_ad0 GND UNUSED VDDQ p2d_d0_id0 GND d2p_d0_id5 VDD fotx0_txd9 GND UNUSED VDDQ UNUSED GND fotx0_txd3 VDD fotx0_txd0 GND
17
UNUSED ibpen0 jtag_tms crdcrcen0 jtag_tck d2p_d0_id6 jtag_trst_L d2p_d0_id1 p2d_d0_id4
18
GND UNUSED VDDQ oob0_int_lo GND d2p_d0_id4 VDD p2d_d0_id7 GND
19
oob0_int_hi d2p_d0_id7 jtag_tdi crden0 UNUSED d2p_d0_id3 UNUSED p2d_d0_id2 d2p_d0_od7
A B C D E F G H J K L M N P R T U V W Y AA AB AC AD AE
p2d_d0_id5 d2p_d0_od5 d2p_d0_od4 ce0_scan UNUSED lssd_ce1_a d2x_d0a_en x2d_d0a_en x2d_d0b_e x2d_d0b_cn d2x_d0b_e VDDQ UNUSED GND UNUSED VDD d2x_d0a_o GND x2d_d0a_e VDDQ
d2p_d0_od6 d2p_d0_od3 d2p_d0_id0 d2p_d0_od0 UNUSED UNUSED UNUSED fotx0_txd2 forx0_rxd8 forx0_rxd6 forx0_rxd5 lssd_ce1_b forx0_rxd1 lssd_ce1_c1 pin_diode0_ reset_L lssd_ce1_c2 forx0_clk lssd_scan_ in1 VDDQ UNUSED GND UNUSED VDD forx0_rxd9 GND x2d_d0a_o VDDQ forx0_rxd2 GND vcsel0_ reset_L VDD pin_diode0_ sig_det GND p2d_d0_id3 UNUSED UNUSED UNUSED d2x_d0a_cn x2d_d0a_on forx0_rxd7 d2x_d0a_c UNUSED forx0_rxd4 UNUSED forx0_rxd3 lssd_ce1_c3 forx0_rxd0 vcsel0_ laser_act
d2x_d0b_on d2x_d0b_en x2d_d0b_en x2d_d0a_cn mon9 mon14 lssd_scan_ in5 x2d_d0b_o lssd_scan_ in4 GND xcvr0_ loopback VDD mon15 GND lssd_scan_ in0 x2d_d0b_on lssd_scan_ in3 VDDQ xcvr0_sig_ det GND
xcvr0_com_ d2x_d0b_cn det lssd_scan_ VDDQ in2
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
143
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 18.
Dataslice Alpha Pin List
Signal Name
ce0_io ce0_scan ce0_tstm3 crdcrcen0 crdcrcen1 crden0 crden1 d2p_d0_id0 d2p_d0_id1 d2p_d0_id2 d2p_d0_id3 d2p_d0_id4 d2p_d0_id5 d2p_d0_id6 d2p_d0_id7 d2p_d0_od0 d2p_d0_od1 d2p_d0_od2 d2p_d0_od3 d2p_d0_od4 d2p_d0_od5 d2p_d0_od6 d2p_d0_od7 d2p_d1_id0 d2p_d1_id1 d2p_d1_id2 d2p_d1_id3 d2p_d1_id4 d2p_d1_id5 d2p_d1_id6 d2p_d1_id7 d2p_d1_od0 d2p_d1_od1 d2p_d1_od2 d2p_d1_od3 d2p_d1_od4 d2p_d1_od5 d2p_d1_od6 d2p_d1_od7 d2x_d0a_c d2x_d0a_cn d2x_d0a_e d2x_d0a_en d2x_d0a_o d2x_d0a_on d2x_d0b_c d2x_d0b_cn d2x_d0b_e d2x_d0b_en d2x_d0b_o d2x_d0b_on d2x_d1a_c d2x_d1a_cn d2x_d1a_e d2x_d1a_en d2x_d1a_o d2x_d1a_on d2x_d1b_c d2x_d1b_cn d2x_d1b_e d2x_d1b_en
Pin
M15 M13 P15 D17 B02 D19 D02 K19 H17 F15 F19 F18 H16 F17 B19 L17 M12 J15 K18 L13 K14 K17 J19 B01 F03 F02 H04 D01 F01 H06 H03 H05 J01 K03 J05 K06 K02 K05 L03 V19 R19 V15 R13 T14 U15 AD12 AD15 W13 Y14 V12 Y13 R01 V01 Y04 T07 R07 V05 T08 W07 AB05 AD07
Signal Name
d2x_d1b_o d2x_d1b_on forx0_clk forx0_rxd0 forx0_rxd1 forx0_rxd2 forx0_rxd3 forx0_rxd4 forx0_rxd5 forx0_rxd6 forx0_rxd7 forx0_rxd8 forx0_rxd9 forx1_clk forx1_rxd0 forx1_rxd1 forx1_rxd2 forx1_rxd3 forx1_rxd4 forx1_rxd5 forx1_rxd6 forx1_rxd7 forx1_rxd8 forx1_rxd9 fotx0_txd0 fotx0_txd1 fotx0_txd2 fotx0_txd3 fotx0_txd4 fotx0_txd5 fotx0_txd6 fotx0_txd7 fotx0_txd8 fotx0_txd9 fotx1_txd0 fotx1_txd1 fotx1_txd2 fotx1_txd3 fotx1_txd4 fotx1_txd5 fotx1_txd6 fotx1_txd7 fotx1_txd8 fotx1_txd9 fotx_clk_in fotx_clk_out GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND
Pin
Y06 V07 AD17 AD19 Y17 Y18 AB19 Y19 V17 U17 U19 T17 T18 AD03 T06 T02 T03 U01 U03 V03 Y01 AB01 Y02 Y03 V16 P12 R17 T16 T15 R15 N11 M10 L15 K16 L05 K04 R05 N09 T05 T04 U05 V04 R03 P08 U09 V09 A02 A06 A10 A14 A18 C04 C08 C12 C16 E02 E06 E10 E14 E18 G04
Signal Name
GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND ibpen0 ibpen1 jtag_tck jtag_tdi jtag_tdo jtag_tms jtag_trst_L lssd_ce1_a lssd_ce1_b lssd_ce1_c1 lssd_ce1_c2 lssd_ce1_c3 lssd_scan_in0 lssd_scan_in1 lssd_scan_in10
Pin
G08 G12 G16 J02 J06 J10 J14 J18 L04 L08 L12 L16 M09 N02 N06 N10 N14 N18 P11 R04 R08 R12 R16 U02 U06 U10 U14 U18 W04 W08 W12 W16 AA02 AA06 AA10 AA14 AA18 AC04 AC08 AC12 AC16 AE02 AE06 AE10 AE14 AE18 B17 B03 E17 C19 E15 C17 G17 P13 W17 AA17 AC17 AC19 AA15 AE17 AC07
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
144
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Signal Name
lssd_scan_in11 lssd_scan_in12 lssd_scan_in13 lssd_scan_in14 lssd_scan_in15 lssd_scan_in2 lssd_scan_in3 lssd_scan_in4 lssd_scan_in5 lssd_scan_in6 lssd_scan_in7 lssd_scan_in8 lssd_scan_in9 lssd_scan_out0 lssd_scan_out1 lssd_scan_out10 lssd_scan_out11 lssd_scan_out12 lssd_scan_out13 lssd_scan_out14 lssd_scan_out15 lssd_scan_out2 lssd_scan_out3 lssd_scan_out4 lssd_scan_out5 lssd_scan_out6 lssd_scan_out7 lssd_scan_out8 lssd_scan_out9 mon0 mon1 mon10 mon11 mon12 mon13 mon14 mon15 mon2 mon3 mon4 mon5 mon6 mon7 mon8 mon9 No Pin oob0_int_hi oob0_int_lo oob1_int_hi oob1_int_lo oob_ad0 oob_ad1 oob_ad2 oob_ad3 oob_ad4 oob_ad5 oob_ad6 oob_ad7 oob_clk oob_devsel0
Pin
AE07 AC05 AE05 AE03 AA05 AE15 AC15 AE13 AC13 W11 AA11 AA09 W09 AC01 AC03 E11 G11 C13 A13 C15 A15 AA03 W03 P07 P05 M05 M07 G09 E09 AD06 AB07 U11 AB12 R11 T11 AB13 AD14 T09 R09 AB08 T10 P10 Y12 V11 AA13 A01 A19 D18 D04 D03 B16 F13 G13 B15 D14 H12 E13 B14 D06 H07
Signal Name
oob_devsel1 oob_devsel2 oob_valid_L oob_wait_L p2d_d0_id0 p2d_d0_id1 p2d_d0_id2 p2d_d0_id3 p2d_d0_id4 p2d_d0_id5 p2d_d0_id6 p2d_d0_id7 p2d_d1_id0 p2d_d1_id1 p2d_d1_id2 p2d_d1_id3 p2d_d1_id4 p2d_d1_id5 p2d_d1_id6 p2d_d1_id7 p2d_ic0 p2d_ic1 p2d_ic2 p2d_ic3 p2d_ic4 p2d_ic5 p2d_ic6 p2d_ic7 p2d_oc0 p2d_oc1 p2d_oc2 p2d_oc3 p2d_oc4 p2d_oc5 p2d_oc6 p2d_oc7 pin_diode0_reset_L pin_diode0_sig_det pin_diode1_reset_L pin_diode1_sig_det plllock plltest_in plltest_out pwrup_reset_L ref_clk ref_clkn soc_in soc_inn test_di1 test_di2 test_lt test_re test_ri UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED
Pin
K08 B04 B08 B05 F16 H15 H19 L19 J17 K13 G15 H18 K01 H02 F04 G05 J03 L01 H01 L07 E07 J09 D08 L09 F08 F07 H08 D07 L11 D13 J11 F12 D12 H11 H09 K10 AB17 AD18 AB03 AD02 E05 E03 G03 B06 C03 C01 G07 F06 A05 A03 C07 C05 A07 A09 A11 A17 B07 B09 B10 B11
Signal Name
UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED
Pin
B12 B13 B18 C09 C11 D05 D09 D10 D11 D15 D16 E19 F05 F09 F10 F11 F14 G01 G19 H10 H13 J07 J13 K12 M01 M02 M03 M04 M06 M14 M16 M17 M18 M19 N01 N03 N05 N07 N13 N15 N17 N19 P01 P02 P03 P04 P06 P14 P16 P17 P18 P19 V10 W01 W19 Y09 Y10 Y11 AA01 AA19
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
145
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Signal Name
UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED vcsel0_laser_act vcsel0_reset_L vcsel1_laser_act vcsel1_reset_L VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ
Pin
AB09 AB10 AB11 AC09 AC11 AD01 AD09 AD10 AD11 AE09 AE11 AE19 AB18 AE01 AB02 A08 A12 C06 C14 E08 G02 G10 G18 J04 J12 J16 L10 M11 N08 N12 P09 R02 R06 R10 R14 R18 U04 U12 U16 W10 AA08 AC02 AC06 AC14 AC18 AE08 AE12 C18 E16 L14 L18 N16 W18 AA16 U08 W06 W14 AA12 AC10 AE04
Signal Name
VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDA Vref Vref Vref Vref Vref Vref x2d_d0a_c x2d_d0a_cn x2d_d0a_e x2d_d0a_en x2d_d0a_o x2d_d0a_on x2d_d0b_c x2d_d0b_cn x2d_d0b_e x2d_d0b_en x2d_d0b_o x2d_d0b_on x2d_d1a_c x2d_d1a_cn x2d_d1a_e x2d_d1a_en x2d_d1a_o x2d_d1a_on x2d_d1b_c x2d_d1b_cn x2d_d1b_e x2d_d1b_en x2d_d1b_o x2d_d1b_on xcvr0_com_det xcvr0_loopback xcvr0_sig_det xcvr1_com_det xcvr1_loopback xcvr1_sig_det
Pin
AE16 C02 E04 L02 L06 N04 W02 AA04 A04 A16 C10 E12 G06 G14 J08 E01 M08 K07 H14 K15 K09 K11 W15 Y16 V14 T13 V18 T19 T12 V13 U13 Y15 AD13 AB15 W05 Y05 V06 U07 T01 V02 AD05 AD08 V08 Y08 Y07 AA07 AD16 AB14 AB16 AD04 AB06 AB04
146
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
2.5.2
Package Dimensions
This section outlines the mechanical dimensions for the Dataslice device. The package is a 474 ceramic ball grid array (CBGA). NOTE: Drawings are not to scale.
Figure 56. Dataslice CBGA Package Dimensions - Top and Side Views
D D1
E1
E
Top View
Pin A01 corner
A
A1
A2
aaa
Side View
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
147
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 57. Dataslice CBGA Package Dimensions - Bottom View
b
e
AE AD AC AB AA Y W V U T R P N M L K J H G F E D C B A 02 01 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19
Pin A01 corner Bottom View
Table 19.
Dataslice CBGA Mechanical Specifications
e =1.27 mm Symbol Minimum A A1 A2 aaa D D1 E E1 M N b 0.82 NOTE: Ball alloy is 90/10 PbSn Nominal 5.22 4.32 0.90 0.15 25.00 22.86 32.50 30.48 19 x 25 474 0.93 mm Maximum mm mm mm mm mm mm mm mm Units
148
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
2.5.3
Part Number
The PMC-Sierra Part Number for the Dataslice is: Dataslice PM9313-HC
PMC Logo Source "B" Part Number
O 6 K22 9 8 P Q P M9 31 3 -HC D a ta sl i ce Rev B BXXLLLLLLLL
Country of Origin
PMC Part Number Device Name Revision Lot Number Date Code
B Y Y WW X X XX X XX X XX X X
Terminal A01 Identifier
PM 9 3 1 3 - H C
PMC prefix IRD prefix IC Chipset family code, 1=TT1, 5 =TTX Sequential part number (1=SCH, 2=XBAR, 3=DS, 5=EPP) Temp. range (C = Commercial 0-70 degrees C) Package code (U = CCGA, H = CBGA)
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
149
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
150
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3
Enhanced Port Processor
This chapter contains information on the Enhanced Port Processor device, part number PM9315-HC, available from PMC-Sierra, Inc. The Enhanced Port Processor (EPP) is based on the original TT1TM Port Processor, but has been greatly improved in terms of its ability to handle OC-48c channels. The original Port Processor (PP) could only support OC-48c at a single priority and required all linecards to be OC-48c. The EPP can support OC-48c at the four priority levels and can be used in OC-48c or OC-192c mode on a port-by-port basis. This enables the EPP to be used with the rest of the ETT1 Chip Set and to provide up to 128 ports of OC-48c. Also, multicast handling has been substantially improved over the PP; now the EPP can handle multicast at the OC-48c port level. However, the most important feature of the EPP is that it operates with the original TT1 Dataslice, Crossbar and Scheduler devices. The main features of the EPP are: * * Four ports of OC-48c or one port of OC-192c. Communicates with other EPPs (not PPs) which can be connected either to OC-48c linecards or OC-192c linecards. Supports four priority levels for unicast and multicast for OC-48c or OC-192c. Supports TDM channels for OC-48c and OC-192c. Uses the LCS (Linecard to Switch) protocol. NOTE: The EPP itself operates on an OC-192c rate stream; therefore, a multiplexer function is needed for OC-48c mode. This multiplexer sits between the four OC-48c rate linecards and the ETT1 port card, and maps the four incoming OC-48c rate streams onto a single stream to the EPP. Each request label contains the information needed to identify its source OC-48c linecard. Figure 58 shows the interconnection of the EPP to other ETT1 devices.
* * *
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
151
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 58. An ETT1 Port Operating in Sub-port Mode with Four OC-48c Linecards
ETT1 Port board OC-48c 0 Dataslice 0/1 Dataslice 0/1 Dataslice 0/1 Dataslice 0/1 Dataslice 0/1 Dataslice 0/1 25M cell/s interface OC-48c 3 Scheduler EPP Flow Control Crossbar Crossbar Crossbar Crossbar Crossbar Crossbar Crossbar MUX
ETT1 Port board
OC-48c 1
OC-48c 2
152
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.1
EPP DATA FLOWS AND BLOCKS
This section describes the functionality of the EPP and its basic structure.
Figure 59. Functional Diagram of LCS Enhanced Port Processor
Input EPP Grant/Data to iDS Request/Data from iDS p2d_d[2:0]_id d2p_d[2:0]_id Control to iDS p2d_d[6:0]_ic Output EPP Control to oDS Data from oDS p2d_d[6:0]_oc d2p_d[1:0]_od
LCS Grant Manager
Input Queue Manager
Output Queue Manager
Output Scheduler
Grant Matching oTIB TDM Frame Tables iTIB
Scheduler Request Modulator
Grant from Scheduler s2p_s[1:0]
Request to Scheduler p2s_s[1:0]
MC Backpressure to Scheduler p2s_s[1:0] Routing Tag from Scheduler s2p_s[1:0]
UC Backpressure from FlowControl Crossbars f2p_[ab][1:0]
UC Backpressure to FlowControl Crossbars p2f_[ab][1:0]
core clocks ref_clk(n) plllock
OOB Interface Control/Status Registers
JTAG
PLL
oob_int_hi
oob_int_lo oob_devsel[3:0]
oob_clk oob_ad[7:0] oob_valid_L
soc_in(n) soc_out
oob_wait_L
jtag_tdi jtag_tms jtag_tck jtag_trst_L jtag_tdo
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
153
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
The EPP is logically split up into an input/ingress EPP (iEPP) and an output/egress EPP (oEPP). The LCS Grant Manager sends grants for input queues which have some space left, and arbitrates among competing flows. The Grant Matching logic matches incoming cell bodies with their LCS grant labels. The iEPP Tag Information Base (iTIB) is a table of multicast port fanouts indexed by LCS multicast tags. The TDM Frame Tables are indexed by TDM slot and contain scheduling information. The Scheduler Request Modulator provides OC-48c granularity backpressure for unicast flows, arbitrates among competing best-effort flows, and restores OC-48c granularity to port-granularity grants from the Scheduler chip. The Output Scheduler block arbitrates among all egress flows to determine which flow may send a cell to the linecard each cell time. The following section describes cell flows within the EPP. The remaining sections describe the blocks of the EPP in more detail.
3.1.1
3.1.1.1
EPP Data Flows
iEPP Cell Arrival: LCS Request-Grant-Cell Flow
The EPP uses the LCS, version 2 (Linecard-to-Switch) protocol. This is a major enhancement to the original LCS protocol in that requests from the linecard are separated from their cell body, reducing the storage requirements of ETT1. The details of LCS are described elsewhere, but the basic operation is as follows: 1. A cell arrives at the ingress linecard and is destined for some output channel, O. 2. The linecard issues a request to the EPP, indicating the output queue (O). The cell body is not sent to the EPP at this time. 3. The EPP returns a grant to the linecard, indicating that the cell body can now be forwarded to the switch. This grant is also an implicit credit, so that the linecard can issue a further request for this output (O). The input Dataslices, iDS0, iDS1 and iDS2, send their input request and cell data to the input EPP on d2p_d0_id[7:0], d2p_d1_id[7:0], d2p_d2_id[7:0], respectively. The LCS ingress header and first 10 bytes of cell payload are sent. The request portion of the LCS ingress header goes to the LCS Grant Manager, which arbitrates among all flows with requests (and space in the input queues) to choose the flow which will receive the next grant. The LCS Grant Manager sends the grant label and sequence number back to input Dataslice 0 via p2d_d0_id[7:0]. If the grant is unicast, its label information is put into a programmable delay line to the Scheduler Request Modulator. This allows requests to be sent to the Scheduler before the arrival of unicast cell bodies. The delay is programmed so that Scheduler grants will not come back before the arrival of the cell bodies, providing that the system round trip time constraint (64 celltimes) is met. The LCS Grant Manager also sends the grant label and sequence number to the Grant Matching block, which matches incoming cell sequence numbers with previously stored grant sequence numbers, and restores the associated label information to the cell whose payload has just arrived. When a match is made, several things happen:
154
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
*
The cell's label information is sent to the input Queue Manager, which determines where the cell will be stored in input Dataslice queue memory. If the flow's queue is already full, the cell is dropped and an interrupt is asserted. Otherwise, the EPP sends a write command & write address to all input Dataslices via p2d_d[6:0]_ic[7:0]. At the same time, the EPP sends egress cell label information to the input Dataslices, via p2d_d0_id[7:0] and p2d_d1_id[7:0]. This information is written into the egress cell header location of the cell. If the cell is multicast, its tag (from the label) is used as the index into the iTIB, to look up the port fanout. If the fanout is all-0's, the cell is dropped. Otherwise the fanout is sent to the Scheduler Request Modulator for arbitration with other unicast and multicast cells waiting to be requested.
*
3.1.1.2
iEPP Cell Departure: Scheduler Request-Grant Flow
Unicast and multicast requests to the Scheduler come from the Scheduler Request Modulator, while TDM requests to the Scheduler come from the TDM Frames block. Requests are sent to both (redundant) Schedulers, p2s_s[1:0]. The Scheduler Request Modulator receives unicast egress queue occupancy information from all EPPs' Output Schedulers via the Flow Control Crossbars, f2p_[a:b][1:0]. It uses this information to decide whether it can pass on requests to the Scheduler for cells destined for those queues. The Scheduler Request Modulator also keeps track of subport information for each request sent to the Scheduler, and restores the subport information to grants coming back from the Scheduler via s2p_s[1:0]. This is because the Scheduler only supports port-granularity in its arbitration. When subport information has been restored to a valid Scheduler grant, the grant info is sent to the Input Queue Manager. If the specified input queue is empty, then an interrupt is asserted and no cell is sent. Otherwise, the Input Queue Manager generates a read command and read address, sent to the input Dataslices via p2d_d[6:0]_ic[7:0]. The input Dataslices then send the cell across the Data Crossbars. The Scheduler also sends a reverse routing tag to the EPP, to indicate from which port this port will be receiving the next cell. This is passed to the input Dataslices via p2d_d[6:0]_ic[7:0]. The input Dataslices pass the tag on to the data Crossbars, to configure the Crossbar fabric.
3.1.1.3
oEPP Cell Arrival: Data Crossbar to oEPP Flow
After a cell traverses the data Crossbars, it is sent out to the output Dataslices. oDS0 and oDS1 send their cell data to the oEPP via d2p_d0_od[7:0] and d2p_d1_od[7:0]. The Output Queue Manager determines which output queue the cell is destined for, and generates a write command and write address for the output Dataslices. This is sent via p2d_d[6:0]_oc[7:0]. The Output Scheduler is notified that a cell has been added to that output queue. If the incoming cell causes a multicast output queue to grow larger than the programmable backpressure threshold, then backpressure is asserted to the Scheduler via p2s_s[1:0]. Optionally, queue size can be ignored and arriving multicast cells will simply be dropped if the queue fills up.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
155
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
The Scheduler's backpressure mechanism is not used for unicast because the Output Scheduler on each port provides the Scheduler Request Modulator on each port with subport-granularity queue information via the Flow Control Crossbars. The Scheduler's backpressure mechanism is not used for TDM because TDM programming should ensure that the output queues cannot overflow. When multicast cells arrive at an oEPP in OC-48c mode, the source port and multicast tag are used to index the oTIB, to look up the egress subport fanout. The egress subport fanout for TDM cells is taken from the TDM Frame table entry for the slot whose cells are currently arriving at oEPPs.
3.1.1.4
oEPP Cell Departure: oEPP to Linecard Flow
The Output Scheduler arbitrates among all flows with cells waiting to be sent to the linecards. When the oEPP is in OC-48 mode, it arbitrates for each output linecard in turn (subport 0, subport 1, subport 2, subport 3, subport 0,...). When the Output Scheduler decides to send a cell, it passes label information to the Output Queue Manager which sends a read command and read address to the Output Dataslices via p2d_d[6;0]_oc[7:0]. When a multicast queue size falls below the programmable unbackpressure threshold, unbackpressure is sent to the Scheduler. The Output Scheduler sends incremental credits for unicast egress queues to all EPPs' Scheduler Request Modulators via p2f_[a.b][1:0].
3.1.2
LCS Grant Manager
The purpose of the LCS Grant Manager is to prevent any input queue from overflowing, and to arbitrate among competing flows. It consists of linecard request counters, input queue debit counters, and an arbiter.
3.1.2.1
Linecard Request Counters
Each linecard request counter is initialized to 0 at reset, incremented when a new LCS Request arrives for the corresponding flow, and decremented when the LCS Grant Manager sends an LCS Grant to the requesting linecard. Request counters can also be updated by LCS Control Packet or OOB access. They are stored in the Linecard Request Counters memory. See Section 3.4 "Enhanced Port Processor Registers" for detailed address and data formats of this memory. These counters are maintained within the EPP and no customer interaction is required. However, customers may choose to monitor request counts on various flows, or to implement request count updates using this memory instead of LCS Control Packets. For Control Packet and TDM queues, there is always one request counter per input queue. For multicast and unicast, if the EPP is in OC-48c mode, there are four request counters per input queue, one for each
156
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
subport. The LCS Grant Manager arbiter ensures that no request counter can be starved by the others; this is described in Section 3.1.2.3 "LCS Grant Manager Arbitration". The size of the request counters for TDM, multicast and unicast flows allows for request counts much larger than the input queue sizes, in order to avoid blocking requests from the input linecards. Maximum request counter values depend on whether the EPP is in OC-48c mode, and are shown Table 20.
Table 20. Linecard Request Count Maximum Values
Traffic Type TDM Multicast Unicast Control Packets
OC-48 port Max Request Count 255 255 255 7
OC-192c port Max Request Count 1023 1023 1023 7
Control packet requests have the highest priority of all traffic types, are always considered to have sufficient queue space, and are usually granted immediately. However, when the 1-in-N Idle Counter or 1-in-N NoGrant Counter expires, the LCS Grant Manager is prohibited from sending a grant during that celltime. To allow some margin for these cases, the control packet request counters are three (3) bits wide instead of one (1)bit. When a TDM Sync command arrives from the Scheduler, all TDM request counters are immediately reset to 0. If the request counters weren't already 0, the "Linecard Requests Flushed by TDM Sync" interrupt is raised. If a linecard sends too many requests for some flow and overflows that request counter, then the request counter will stick at the maximum value and the "Linecard Request Count Overflow" interrupt will be raised. Multicast, TDM and CP tags from LCS Request labels must be stored in FIFOs corresponding to the request counters, since these tags must be returned to the linecards as part of LCS Grant labels. These FIFOs are contained in the Linecard Multicast Tags memory, the Linecard TDM Tags memory, and some auxiliary registers accessed along with CP request counters in the Linecard Request Count memory. These FIFOs are maintained within the EPP and no customer interaction is required.
3.1.2.2
Input Queue Debit Counters
Each input queue has one debit counter which is initialized to 0 at reset, incremented when the EPP sends an LCS Grant for that queue to a linecard, and decremented when the Scheduler sends a grant for that queue. Control Packet queues are always considered to have 0 debits; Control Packet requests will always be immediately granted. So there are no debit counters for Control Packet input queues. Linecards must make sure to not overflow the Control Packet input queues (only CPU Control Packets go to the input queues; LCS Processed Control Packets are processed and dropped immediately).
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
157
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.1.2.3
LCS Grant Manager Arbitration
The LCS Grant Manager arbiter makes a linecard grant decision every celltime, based on information from the Linecard Request Counters and Input Queue Debit Counters. When the EPP is in OC-48c mode, each queue is shared by 4 request counters, one per subport. Therefore, in OC-48c mode, additional state information about multicast and unicast flows is used in the decision, in order to prevent any request counter from being starved by the other request counters contending for the same queue. When the EPP is in OC-48c mode, the LCS Grant Manager arbitrates between the eligible flows requested by each subport in turn (subport 0, subport 1, subport 2, subport 3, subport 0,...). Thus each OC-48c linecard will receive at most one grant every 4 celltimes. A flow is eligible for arbitration if its request counter is nonzero and its debit counter is less than the queue size. The arbiter enforces the ETT1 service class priority scheme (Control Packets > TDM > MC0 > UC0 > MC1 > ... > UC3). Within each unicast priority, the arbiter performs a round-robin on eligible flows. The order of flows considered in the round-robin is increasing {egress_subport, egress_port}. So for an egress port that is in OC-48c mode, its subports will appear once each 32 steps in the ordering. For an egress port in OC-192c mode, there is only one valid subport (0), but each egress_subport step for that port will map to subport 0. Because of the strict order in which grants are given to each subport in OC-48c mode, and the unpredictable timing of cells leaving a congested queue, it would be possible for some request counters to never receive grants if the arbitration were left as a free-for-all. In order to avoid this, the LCS Grant Manager "reserves" the last space in each unicast or multicast queue for a rotating favorite subport. When a debit counter indicates that there is only one space left in a queue, only the request counter of the favorite subport for that queue is eligible. Whenever the favorite subport gets a grant, regardless of whether the queue is down to its last space, the favorite subport changes to the next subport with a non-zero request count. This algorithm ensures that no request counter can be starved.
3.1.3
Scheduler Request Modulator
The purpose of the Scheduler Request Modulator is to prevent unicast output queues from overflowing. The Scheduler's backpressure mechanism only supports port-granularity backpressuring, which is not acceptable for unicast (subport-granularity) queues. So instead of sending backpressure commands from the oEPP to the Scheduler, all oEPPs send queue occupancy information to all iEPPs via the Flow Control Crossbars. The Scheduler Request Modulator uses the output queue occupancy information to prevent overflowing any unicast output queue; if there is not enough room left in an output queue, no more requests for that output queue will be sent to the Scheduler until more room becomes available. Since unicast and multicast are interleaved per-priority in the overall ETT1 priority scheme, multicast requests are also arbitrated by the SRM. The SRM consists of input queue request counters, output queue debit counters, an arbiter for deciding which flow to request to the Scheduler, and state for restoring subport information to Scheduler grants.
3.1.3.1
Input Queue Request Counters
Input Queue request counters are initialized to 0 at reset. Unicast and multicast request counters behave slightly differently.
158
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Unicast request counters are incremented when unicast request/grant labels arrive via a programmable delay line from the LCS Grant Manager. Using the programmable delay line instead of waiting for unicast cells to arrive allows unicast requests to be sent to the Scheduler ahead of arrival so that Scheduler Grants can arrive at the same time as the cells, effectively masking Scheduler latency from the round trip time. The delay line is programmed so that Scheduler grants cannot arrive before the cells arrive, provided that the system round trip time constraint is met (64 celltimes from the grant to the cell arrival). Multicast request counters are incremented when the Grant Matching block matches arriving multicast cells with their request/grant labels. Unicast and multicast request counters are decremented as requests are sent to the Scheduler.
3.1.3.2
Output Queue Debit Counters
Output queue debit counters are initialized to 0 at reset. They are incremented when unicast requests are sent to the Scheduler. When incremental output queue credits arrive via the Flow Control Crossbars, they are subtracted from the appropriate debit counters. The counters are contained in the Output Queue Debit Counters Memory. See Section 3.4 "Enhanced Port Processor Registers" for detailed address and data formats. These counters are maintained within the EPP and no customer interaction is required, unless the customer chooses to implement the Flow Control recovery procedure outlined in "Enhanced Port Processor - Flow Control Crossbar" on page 94. Output queue debit counts are maintained for unicast only, not multicast, since the Scheduler's backpressure mechanism can be used to prevent multicast output queues from overflowing.
3.1.3.3
Scheduler Request Count Modulator Arbiter
The arbiter decides every celltime which best-effort flow, if any, to request to the Scheduler. A unicast flow is eligible for arbitration if its input queue request counter is nonzero and its output queue debit counter is less than the queue size. A multicast flow is eligible for arbitration if its request count is nonzero and if fewer than 64 requests for that multicast flow are currently waiting for grants from the Scheduler.. Multicast flows are included in the arbitration in order to enforce the ETT1 best-effort service class priority scheme (MC0 > UC0 > MC1 > ... > UC3), and to prevent 96-cell multicast input queues from exceeding the maximum Scheduler request count of 64. Within each unicast priority, a round-robin is used to arbitrate among eligible flows. The order of the round-robin is increasing (egress_subport, egress_port), like the LCS Grant Manager arbiter.
3.1.4
Input Queue Manager
The Input Queue Manager stores arriving cells when the Grant Matching block has restored their grant label information, and sends departing cells when the Scheduler Request Modulator restores subport granularity to Scheduler grants. Cells are dequeued only when the Scheduler sends a dequeue command along with a grant.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
159
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Unicast input queues are per-priority, per-destination (port, subport). If the destination port is OC-192c, the only valid subport is 0 and the queue length is 64 cells. If the destination port is OC-48c, there is a separate queue per subport, 16 cells each. Multicast input queues are per-priority and are 96 cells in length. TDM input queues are per-subport. If this port is OC-192c, the only valid subport is 0 and the queue length is 96. If this port is OC-48c, there is a separate queue per subport, 24 cells each. When a TDM Sync command arrives from the Scheduler, TDM input queues are instantly emptied. If cells were present in any TDM input queue, the "Cells Flushed by TDM Sync" interrupt is raised. Control Packet input queues are per-subport. The length is 8 regardless of OC-48c mode. Control Packet queues never receive grants from the Scheduler. Instead, OOB must write (any value) to the appropriate "Linecard to OOB FIFO Subport [3:0] Status" register in order to dequeue a Control Packet cell. If a Control Packet arrives for a full queue, an "LC2OOB/CPU CP FIFO Overflow, LC[3:0]" interrupt is raised. If the Scheduler grants to an empty (unicast/multicast/tdm) queue, a "Scheduler Grant to Empty * Queue" is raised. If a cell of any traffic type arrives for a full queue, the cell will be dropped (no write command to the Dataslices). If there is a Scheduler grant to any empty queue, no cell will be transmitted (no read command to the Dataslices). All input queues are simple circular buffers.
3.1.5
Output Queue Manager
The Output Queue Manager stores arriving cells from the Data Crossbars, and sends/dequeues departing cells under control of the Output Scheduler. Unicast output queues are per-priority, per-source port, per-destination subport. If this EPP is OC-192c, the only valid subport is 0, and the length is 64. If this EPP is OC-48c, there is a separate output queue for each subport, and the length is 16. All unicast output queues are simple circular buffers. Multicast output queues are per-priority, shared among output subports. When a multicast cell arrives, it is stored at one location regardless of how many destination subports it is to be sent to. In OC-48c mode, a separate list of queue locations is maintained for each subport, so that each subport appears to have a different queue with up to 96 locations. Since the 96 locations are actually shared by all subports, it is possible for congestion on one subport to back up the entire queue. If multicast backpressure is enabled, then multicast traffic destined for other ports at the same priority may be blocked. The multicast output queues are not simple circular buffers, instead they use a linked-list representation. There is only one TDM output queue, length 96, shared among subports the using the same mechanism as the multicast output queues. There is no queue for egress Control Packets; instead, there is one allocated cell location in Dataslice Output Queue Memory for each type of egress Control Packet. Several of these locations must be initialized by the OOB before the switch begins to operate; see Section 3.3 "Output Dataslice Queue Memory Allocation with EPP" for details.
160
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
The Output Queue Manager maintains head and length information for unicast queues in the Output Unicast Queue Information Memory, which is documented in Section 3.4 "Enhanced Port Processor Registers". This information is maintained within the EPP and no customer interaction is required. However, in order to implement the Flow Control recovery procedure outlined in "Enhanced Port Processor - Flow Control Crossbar" on page 94, the customer will need to read the length of each affected unicast queue. The Output Queue Manager is the source of the interrupts "Output UC or MC Queue Overflow" and "output TDM Queue Overflow".
3.1.6
Output Scheduler
The purpose of the Output Scheduler is to arbitrate among cells in the output queues waiting to be sent to the output linecard(s). When the EPP is in OC-48c mode, the Output Scheduler arbitrates among cells destined for each subport in turn (subport0, subport1, subport2, subport3, subport0,...). Unicast traffic can be shut off on a per-source-port basis in the Output Scheduler, by the "Freeze Unicast Output Scheduler Flows" register (see Section 3.4 "Enhanced Port Processor Registers"). Hole requests from the linecard are sent to the Output Scheduler. When a hole request for a given priority is valid, the Output Scheduler will not send any unicast or multicast cell of that priority during that celltime. In OC-48c mode, separate hole requests come from each subport, and are delayed until that subport is due to receive a cell. When in LCS Stop mode, the Output Scheduler does not send cells to the linecard. Since the LCS Start/Stop status is per linecard, subports on the same OC-48c mode port could have different Start/Stop status. Even though cells are not sent to the linecard, TDM cells continue to be dequeued from the TDM output queues. This is to prevent one stopped subport from blocking traffic to other subports, since the output TDM/MC queues are shared among subports. If LCS Lossy mode is in effect, then multicast cells will also continue to be dequeued. The Output Scheduler also sends Control Packets to the linecard(s) under control of the "Send OOB2LC Control Packet" register. When the Scheduler sends a TDM Sync command, the Output Scheduler sends a TDM Sync Control Packet to the linecard(s). Several types of egress Control Packet must be written into the correct locations in the Dataslice Output Queue Memory by software before the switch begins to operate; see Section 3.3 "Output Dataslice Queue Memory Allocation with EPP" for details.
3.1.7
OOB Interface and Control/Status Registers
All of the devices have an OOB interface. This interface allows a single local CPU to control and monitor all of the devices within a core fabric. Internally, each device provides registers that can be mapped into the CPU's address space. These registers are described in more detail in Section 3.4 "Enhanced Port Processor Registers".
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
161
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.2
INPUT DATASLICE QUEUE MEMORY ALLOCATION WITH EPP
The following table lists the raw addresses within input Dataslice queue memory for all input queues (VOQs) managed by the iEPP. These addresses apply to each input Dataslice in the port; to read or write a complete cell in input Dataslice queue memory, the raw address must be translated into OOB address offsets, and then those offsets should be read/written for each Dataslice. Priority is the lower 2 bits of the traffic type for Unicast or Multicast queues; Port is 5 bits, and Subport is 2 bits. "x" is a single bit that can be 0 or 1, used to denote offsets within queues. "yy" can only be 00, 01, or 10, and is used to denote offsets within queues that are 24 or 96 cells in length.
Table 21.
Input Dataslice Queue Memory Allocation
Traffic Type Unicast Unicast Multicast TDM TDM CPU CP CPU CP
Type of Ingress Queue Linecard size either (OC-48c egress port) either (OC-192c egress port) either OC-48c OC-192c OC-48c OC-192c 16 64 96 24 96 8 8
Total # Dataslice Queue Mem Address of Mapping Queues 512 128 4 4 1 4 1 0,Priority,Egress Port, Egress Subport,xxxx 0,Priority,Egress Port,xxxxxx 10000,yy,xxxxx,Priority 1000011,yy,xxx,Ingress Subport 1000011,yy,xxxxx 100001111,Ingress Subport,xxx 100001111,00,xxx
Raw Address Range 0x0000-0x1FFF 0x0000-0x1FFF 0x2000-0x217F 0x2180-0x21DF 0x2180-0x21DF 0x21E0-0x21FF 0x21E0-0x21E7
Each raw address given above is for a 48-bit word in Dataslice queue memory. In order to access the memory with OOB, a raw address must be converted into two OOB address offsets because OOB accesses are 32-bit. The following formula is used: oob_offset_16_MSBs = 0x40000 | (raw_address << 3) oob_offset_32_LSBs = 0x40000 | (raw_address << 3) | 0x4
3.3
OUTPUT DATASLICE QUEUE MEMORY ALLOCATION WITH EPP
The following table lists the raw addresses within output Dataslice queue memory for all output queues (VIQs) managed by the oEPP. These addresses apply to each output Dataslice in the port; to read or write a complete cell in output Dataslice queue memory, the raw address must be translated into OOB address offsets, and then those offsets should be read/written for each Dataslice.
162
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
NOTE: There is no output queue for Control Packets. Instead, one cell location is statically allocated for each type of outgoing Control Packet. OOB must initialize the processed CP locations in output Dataslice queue memory before starting EPP operation. Priority is the lower 2 bits of the traffic type for Unicast or Multicast queues; Port is 5 bits, and Subport is 2 bits. "x" is a single bit that can be 0 or 1, used to denote offsets within queues. "yy" can only be 00, 01, or 10, and is used to denote offsets within queues that are 96 cells in length.
Table 22. Output Dataslice Queue Memory Allocation
Traffic Type
Type of Egress Queue Linecard Size OC-48c OC-192c OC-48c OC-192c OC-48c OC-192c OC-48c OC-192c OC-48c OC-192c OC-48c OC-192c OC-48c OC-192c OC-48c OC-192c 16 64 96 96 1 1 1 1 1
Total # of Queues 512 128 4 1 1 1 1 1 1
Dataslice Queue Mem Address Mapping 0,Priority,Ingress Port,Egress Subport,xxxx 0,Priority,Ingress Port,xxxxxx 10000,yy,xxxxx,Priority 1000011,yy,xxxxx 10000111100000 10000111100010 10000111100011 10000111100100 10000111100101
Raw Address Range 0x0000-0x1FFF 0x0000-0x1FFF 0x2000-0x217F 0x2180-0x21DF 0x21E0 0x21E2 0x21E3 0x21E4 0x21E5
Unicast Unicast Multicast TDM CPU (OOB to LC) CP LCS Stop CP LCS Start CP TDM Sync Sel 0 CP TDM Sync Sel 1 CP
Each raw address given above is for a 48-bit word in Dataslice queue memory. In order to access the memory with OOB, a raw address must be converted into two OOB address offsets because OOB accesses are 32-bit. The following formula is used: oob_offset_16_MSBs = 0x60000 | (raw_address << 3) oob_offset_32_LSBs = 0x60000 | (raw_address << 3) | 0x4 The table below shows the data format of egress Control Packets, and the OOB address offsets within each logical Dataslice where data must be written. The values written to shaded locations are left up to the
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
163
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
customer's implementation. The CPU Control Packet locations should be written whenever the customer's software is about to send a CPU control packet to the linecard. The given locations for the other types of Control Packets must be initialized after the Dataslices are reset and before traffic is sent through the switch.
Table 23. Control Packet Type EPP Egress Control Packet Data Format and Dataslice OOB Addressing OOB Address Offset DS0 DS1 DS2 DS3...DS11
CPU (CRC-16 header mode) CPU (CRC-8 header mode) LCS Stop
(msb) 0x70F00 (lsb) 0x70F04 (msb) 0x70F00 (lsb) 0x70F04 (msb) 0x70F10 (lsb) 0x70F14 (msb) 0x70F18 (lsb) 0x70F1C (msb) 0x70F20 (lsb) 0x70F24 (msb) 0x70F28 (lsb) 0x70F2C
0x0000 0x00000010 0x0000 0x00000010 0x0000 0x00000000 0x0000 0x00000000 0x0000 0x00000000 0x0000 0x00000000
0x0031
0x0057
0x0000 0x02000000 0x0000 0x02800000 0x0000 0x00000000 0x0000 0x00800000
0x0000 0x00006E9F 0x0000 0x0000C566 0x0000 0x0000E139 0x0000 0x00004AC0
LCS Start
TDM Sync Sel 0
TDM Sync Sel 1
164
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4
ENHANCED PORT PROCESSOR REGISTERS
The Enhanced Port Processor device select low-order bits are hard wired on the board by four pins (oobdev_sel[3:0]) on the Enhanced Port Processor package. See section 1.7.1.3 "Individual Device Selects" on page 79 for more information.
3.4.1
Enhanced Port Processor Summary
The following table is a summary of information for all registers in the Enhanced Port Processor. See the following Descriptions section for more information on individual registers. Read and Clear means that reading the register causes it to be cleared (reset to zero).
Table 24. Enhanced Port Processor Register Summary
Address 00000h 00004h 00008h 0000Ch 00010h 00014h 00018h 0001Ch 00020h 00024h 00028h 00030h 00034h 00038h
Symbol ESTS ETKNRS EIRLMSK EIRHMSK EIR ELPAIBIM EHPAIBM EAIBIR ELPICIR EHPICIR EIICIR EAIBRS EAIBRDY EEN Status
Register
Access Read Only Read/Write Read/Write Read/Write Read and Clear Read/Write Read/Write Read and Clear Read/Write Read/Write Read and Clear Read/Write Read Only Read/Write
Default Value 51000000h 00000031h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 0000003Fh 00000000h 00000000h
Reset and Control Low Priority Mask High Priority Mask Interrupt Register Low Priority AIB Interrupt Mask High Priority AIB Interrupt Mask AIB Interrupt Register Low Priority Incremental Credit Interrupt Mask High Priority Incremental Credit Interrupt Mask Invalid Incremental Credit Interrupt Register AIB Reset AIB Ready Enable Port Processor
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
165
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 24.
Enhanced Port Processor Register Summary (Continued)
Address 0003Ch 00040h 00044h 00048h 0004Ch 00050h 00054h 00058h 0005Ch 00060h 00064h 00068h 0006Ch 00070h 00074h 00078h 0007Ch
Symbol EAIBEN ELCSSSS ELGCBD ESGEQD ELCSECT ELCSP0CT ELCSP1EC ELCSP2EC ELCSP3EC EIS0VCT EIS1VCT EIS2VCT EIS3VCT EOS0VCT EOS1VCT EOS2VCT EOS3VCT
Register AIB Enable Transmitter LCS Started/Stopped Status/Control Lost Grant or Cell Body Debug Info Scheduler Grant to Empty Queue Debug Info LCS Header CRC Error Count LCS Processor CP Subport 0 CRC Error Count LCS Processor CP Subport 1 CRC Error Count LCS Processor CP Subport 2 CRC Error Count LCS Processor CP Subport 3 CRC Error Count Input Subport 0 Valid Count Input Subport 1 Valid Count Input Subport 2 Valid Count Input Subport 3 Valid Count Output Subport 0 Valid Count Output Subport 1 Valid Count Output Subport 2 Valid Count Output Subport 3 Valid Count
Access Read/Write Read/Write Read and Clear Read and Clear Read and Clear Read and Clear Read and Clear Read and Clear Read and Clear Read and Clear Read and Clear Read and Clear Read and Clear Read and Clear Read and Clear Read and Clear Read and Clear
Default Value 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h
166
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 24.
Enhanced Port Processor Register Summary (Continued)
Address 00080h 00084h 00088h 0008Ch 00090h 00094h 00098h 0009Ch 000A0h 000A4h 000A8h 000ACh 000B0h 000B4h 000B8h 000BCh 000C0h 000C4h 000C8h 000CCh 000DOh 000D4h 00100h 02000-02FFCh
Symbol ES0OOBS ES1OOBS ES2OOBS ES3OOBS ESOBLCP ESTDMSSS EMCIFCT EIDMA EOPBPTHR EEMM ETDMO EFCTL EFUOS EMPD EFTCTRL ERFRSSC ESTRTSC EIDLCT ETDMEN ETDMCTRL ENOGCT EESR48SD EPLLREG ETDMFT0M
Register Subport 0 to OOB FIFO Status Subport 1 to OOB FIFO Status Subport 2 to OOB FIFO Status Subport 3 to OOB FIFO Status Send OOB to Linecard Control Packet Suggested TDM Sync Subport Select Multicast Invalid Fanout Count Internal Delay Matching Adjustments Output Backpressure/Unbackpressure Threshold Egress OC-48/OC-192c Mode Map TDM Offset Value Freeze Control Freeze Unicast Output Scheduler Flows Multicast Priority Drop Fault Tolerance Status/Control Refresh Schedulers Command Go Schedulers Command Idle Count Value Enable TDM Sync Generation TDM Sync Generation Control No Grant Count End of Sch Refresh/Sch OC-48 Sync ModCount PLL Status/Control TDM Frame Table 0 Memory
Access Read/Write Read/Write Read/Write Read/Write Read/Write Read/Write Read and Clear Read/Write Read/Write Read/Write Read/Write Read/Write Read/Write Read/Write Read/Write Write Only Write Only Read/Write Read/Write Read/Write Read/Write Write Only Read/Write Read/Write
Default Value 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 03A8D66Bh 00001C46h 00000000h 00000080h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h FFFFFFFFh 00000000h 0001447Ch 000000000h
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
167
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 24.
Enhanced Port Processor Register Summary (Continued)
Address 03000-03FFCh 04000-07FFCh
Symbol ETDMFT1M EITIBM
Register TDM Frame Table 1 Memory ITIB Memory Subport Request Count Memory Input Queue Information Memory Waiting Scheduler Request Count Memory Output UC Queue Debit Count Memory Output UC Queue Information Memory Output MC Queue Drop Counters OTIB Memory
Access Read/Write Read/Write Read/Write Read/Write Read/Write Read/Write Read/Write Read/Write Read/Write
Default Value 000000000h 000000000h 000000000h 000000000h Unknown 000000000h 000000000h 000000000h 000000000h
08000-0FFFCh ESRCTM 16000-17FFCh EIQIM
1C000-1DFFCh EWSRCT 1E000-1E7FCh EOUCQDCT 20000-21FFCh EOUCQI
3E000-3FFFCh EOMCQDCT 08000-FFFFCh EOTIBM
168
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2
Enhanced Port Processor Register Descriptions
Read and Clear means that reading the register causes it to be cleared (reset to zero). All bits labeled as Reserved should be set to 0.
3.4.2.1 Symbol:
Status ESTS
Address Offset: 00000h Default Value: 51000000h Access: Read Only Bits 31:28 27:24 23:2 1 0 Chip Revision Number. Reserved. High Priority Interrupt. 1 = an outstanding high priority interrupt. One of the bits in the Interrupt Register is set, and is enabled via its corresponding high priority mask. Low Priority Interrupt. 1 = an outstanding low priority interrupt. One of the bits in the Interrupt Register is set, and is enabled via its corresponding low priority mask. Description Chip ID Number. Identifies the specific ETT1 device.
3.4.2.2 Symbol:
Reset and Control ETKNRS
Address Offset: 00004h Default Value: 00000031h Access: Read/Write Reset and Control register. Bits 31-15 14 13:6 5 Reserved. 13th and 14th Dataslice Enable. This bit must be set before bringing up the Dataslices' fiber-optics links in systems that use 14 Dataslices. Note: This bit will read out in bit position 6. Reserved Generate Truncated 16-bit CRC. For Switch to Linecard LCS headers, generate a 16-bit CRC (0x11021) over the header, but only send the lower 8 bits, if this bit is 1. If this bit is 0, generate an 8-bit CRC (0x107) over the header and send that. Description
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
169
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Bits 4
Description (Continued) Check full 16-bit CRC. For Linecard to Switch LCS headers, check a 16-bit CRC (0x11021) over the header, if this bit is 1. If this bit is 0, check an 8-bit CRC (0x107). This applies only to the LCS header and not to Processed Control Packet payload data. Include MuxChip's Subport ID in CRC gen. If this bit is 1, generate the Switch to Linecard LCS header CRC including the MUX field in the grant label. If 0, generate the CRC with the MUX field masked to 0. Include MuxChip's Subport ID in CRC check. If this bit is 1, check the Linecard to Switch LCS header CRC including the MUX field in the request label. If 0, check the CRC with the MUX field masked to 0. Small System Mode. If this bit is set, only Flow Control Crossbar A is used; FC Crossbar B is ignored. Port numbers must be even numbers. This corresponds to the Data Crossbars' 16x16 and 8x8 modes. Reset. Writing a 1 to this location will reset the entire chip. It is equivalent to a hardware reset. This register is cleared automatically when the chip is reset. Soft reset takes 1mS to complete.
3
2
1
0
NOTE: After resetting an EPP, write 0 to offsets 0x1d000 through 0x1d7fc in that EPP (a total of 512 OOB writes). When all EPPs are reset simultaneously (using a broadcast OOB write to the EPP Control register), 512 broadcast OOB writes can be used to reset those locations in all EPPs.
3.4.2.3 Symbol:
Low Priority Mask EIRLMSK
Address Offset: 00008h Default Value: 00000000h Access: Read/Write Interrupt Mask for interrupts. Bits 31:0 Description Low Priority Mask. Mask bits for low priority interrupts. Each mask bit is set to 1 to enable a low priority interrupt when the corresponding bit in the Interrupt Register is 1.
3.4.2.4 Symbol:
High Priority Mask EIRHMSK
Address Offset: 0000Ch Default Value: 00000000h Access: Read/Write
170
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Interrupt Mask for interrupts. Bits 31:0 Description High Priority Mask. Mask bits for high priority interrupts. Each mask bit is set to 1 to enable a high priority interrupt when the corresponding bit in the Interrupt Register is 1.
3.4.2.5 Symbol:
Interrupt Register EIR
Address Offset: 00010h Default Value: 00000000h Access: Read and Clear Interrupt Register. Bits 31 30 29 28 27 26 25 24 23 22 21 20 Description Scheduler Grant to Empty TDM Queue. This is set if a Scheduler grant arrives for a TDM input queue (VOQ) which has no cells waiting. Scheduler Grant to Empty MC Queue. This is set if a Scheduler grant arrives for a MC input queue (VOQ) which has no cells waiting. Cells Flushed by TDM Sync. This is set if a TDM Sync clears waiting cells from TDM input queues. If a TDM Sync arrives and the TDM input queues are already empty, this is not set. Linecard Requests Flushed by TDM Sync. This is set if a TDM Sync clears nonzero TDM Subport request counters. Scheduler Grant to Nonwaiting UC Queue. This is set if a Scheduler Grant arrives for a priority and port for which no subport queues are expecting grants. oDS to oEPP Valid Mismatch. This is set if the three oDS to oEPP frame's valid bit does not match. iDS to iEPP Valid Mismatch. This is set if the three iDS to iEPP frame's valid bit does not match. Grant or Cell Body Dropped. This is set if 128 OC-192c celltimes have passed since a grant, but no cell payload has arrived with the grant sequence number. Invalid Cell Sequence Number from Subport 3. This is sent if Subport 3 has sent a cell payload with a sequence number which does not match any outstanding grant. Invalid Cell Sequence Number from Subport 2. This is sent if Subport 2 has sent a cell payload with a sequence number which does not match any outstanding grant. Invalid Cell Sequence Number from Subport 1. This is sent if Subport 1 has sent a cell payload with a sequence number which does not match any outstanding grant. Invalid Cell Sequence Number from Subport 0. This is sent if Subport 0 has sent a cell payload with a sequence number which does not match any outstanding grant.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
171
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Bits 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3
Description (Continued) Received Multicast Cell with Invalid Fanout. This is set if a multicast cell arrives and its tag's fanout is all-0s. CRC Error in LCS Header from LC??. This is set if an incoming cell has an LCS header CRC error or has a code error in input dataslices 0, 1, or 2. Since the MUX field of the request label could be corrupted, the source subport is unknown. CRC Error in Processed Control Packet from Subport 3. This is set if there is a CRC Error over the first 10 bytes of payload data for a PCP from Subport 3. CRC Error in Processed Control Packet from Subport 2. This is set if there is a CRC Error over the first 10 bytes of payload data for a PCP from Subport 2. CRC Error in Processed Control Packet from Subport 1. This is set if there is a CRC Error over the first 10 bytes of payload data for a PCP from Subport 1. CRC Error in Processed Control Packet from Subport 0. This is set if there is a CRC Error over the first 10 bytes of payload data for a PCP from Subport 0. Received LCS Start Control Packet. This notifies OOB that a Subport has sent a Processed Control Packet of type "LCS Start". Received LCS Stop Control Packet. This notifies OOB that a Subport has sent a Processed Control Packet of type "LCS Stop". Received LC2OOB/CPU Control Packet from Subport 3. This notifies OOB that Subport 3 has sent a CPU Control Packet. Received LC2OOB/CPU Control Packet from Subport 2. This notifies OOB that Subport 2 has sent a CPU Control Packet. Received LC2OOB/CPU Control Packet from Subport 1. This notifies OOB that Subport 1 has sent a CPU Control Packet. Received LC2OOB/CPU Control Packet from Subport 0. This notifies OOB that Subport 0 has sent a CPU Control Packet. LC2OOB/CPU Control Packet FIFO Overflow, Subport 3. This is set if Subport 3 overflows its 8-cell FIFO of CPU Control Packets to OOB. LC2OOB/CPU Control Packet FIFO Overflow, Subport 2. This is set if Subport 2 overflows its 8-cell FIFO of CPU Control Packets to OOB. LC2OOB/CPU Control Packet FIFO Overflow, Subport 1. This is set if Subport 1 overflows its 8-cell FIFO of CPU Control Packets to OOB. LC2OOB/CPU Control Packet FIFO Overflow, Subport 0. This is set if Subport 0 overflows its 8-cell FIFO of CPU Control Packets to OOB. Line Card Request Count Overflow. This is set if a Subport sends too many requests (the counter will stick at max value). Scheduler Grant to Empty UC Queue. This is set if the scheduler grants to an empty UC input queue (VOQ). Output MC or UC Queue Overflow. This is set if a cell from the crossbar overflows a MC or UC output queue. Output TDM Queue Overflow. This is set if a cell from the Crossbar overflows a TDM output queue.
2 1 0
172
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.6 Symbol:
Low Priority AIB Interrupt Mask ELPAIBIM
Address Offset: 00014h Default Value: 00000000h Access: Read/Write Enables low priority interrupts. Bits 31:24 23:0 Reserved. Mask bits for low priority interrupts. Each mask bit is set to 1 to enable a low priority interrupt when the corresponding bit in the AIB Interrupt register is 1. Description
3.4.2.7 Symbol:
High Priority AIB Interrupt Mask EHPAIBM
Address Offset: 00018h Default Value: 00000000h Access: Read/Write Enables low priority interrupts. Bits 31:24 23:0 Reserved. Mask bits for high priority interrupts. Each mask bit is set to 1 to enable a high priority interrupt when the corresponding bit in the AIB Interrupt register is 1. Description
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
173
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.8 Symbol:
AIB Interrupt Register EAIBIR
Address Offset: 0001Ch Default Value: 00000000h Access: Read and Clear Interrupt Register Bits 31:24 23 22 21 20 19 18 17 Reserved. Flow Control Crossbar A1 CRC Error Flow Control Crossbar A0 CRC Error Flow Control Crossbar A1 Ready Down Flow Control Crossbar A0 Ready Down Flow Control Crossbar A1 Ready Up Flow Control Crossbar A0 Ready Up Flow Control Crossbar A Both Links CRC Error. If set, neither A0 nor A1 provided valid incremental credits during at least one celltime, so check the Invalid Incremental Credits interrupt register to see which ports' output queue credits need repair. Flow Control Crossbar A Frame Mismatch. Though apparently valid, the frames received from A0 and A1 were not identical. In this case, the Primary Flow Control Crossbar A is selected. Flow Control Crossbar B1 CRC Error Flow Control Crossbar B0 CRC Error Flow Control Crossbar B1 Ready Down Flow Control Crossbar B0 Ready Down Flow Control Crossbar B1 Ready Up Flow Control Crossbar B0 Ready Up Flow Control Crossbar B Both Links CRC Error. If set, neither B0 nor B1 provided valid incremental credits during at least one celltime, so check the Invalid Incremental Credits interrupt register to see which ports' output queue credits need repair. Flow Control Crossbar B Frame Mismatch. Though apparently valid, the frames received from B0 and B1 were not identical. In this case the Primary Flow Control Crossbar B is selected. Scheduler 1 CRC Error Scheduler 0 CRC Error Scheduler 1 Ready Down Scheduler 0 Ready Down Scheduler 1 Ready Up Scheduler 0 Ready Up Description
16 15 14 13 12 11 10 9
8 7 6 5 4 3 2
174
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Bits 1 0
Description (Continued) Scheduler Both Links CRC Error. If set, traffic for this port is frozen and OOB must initiate Scheduler Refresh to repair Scheduler state for this port. Scheduler Frame Mismatch. Though apparently valid, the frames received from Sched0 and Sched1 were not identical. In this case the Primary Sched is selected.
3.4.2.9 Symbol:
Low Priority Incremental Credit Interrupt Mask ELPICIR
Address Offset: 00020h Default Value: 00000000h Access: Read/Write Enables a low priority interrup Bits 31:0 Description Mask bits for low priority interrupts. Each mask bit is set to 1 to enable a low priority interrupt when the corresponding bit in the Invalid Incremental Credit Interrupt register is 1.
3.4.2.10 Symbol:
High Priority Incremental Credit Interrupt Mask EHPICIR
Address Offset: 00024h Default Value: 00000000h Access: Read/Write Enables a high priority interrupt. Bits 31:0 Description Mask bits for High priority interrupts. Each mask bit is set to 1 to enable a high priority interrupt when the corresponding bit in the Invalid Incremental Credit Interrupt register is 1.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
175
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.11 Symbol:
Invalid Incremental Credit Interrupt Register EIICIR
Address Offset: 00028h Default Value: 00000000h Access: Read and Clear Interrupt register. Bits 31:0 Description Each bit corresponds to a single port. Invalid incremental credits were received from either Flow Control Crossbar A or Flow Control Crossbar B.
3.4.2.12 Symbol:
AIB Reset EAIBRS
Address Offset: 00030h Default Value: 0000003Fh Access: Read/Write Resets AIB link for each port. Bits 31:6 5 4 3 2 1 0 Reserved. Flow Control Crossbar A1 AIB Reset. Flow Control Crossbar A0 AIB Reset. Flow Control Crossbar B1 AIB Reset. Flow Control Crossbar B0 AIB Reset. Scheduler 1 AIB Reset. Scheduler 0 AIB Reset. Description
176
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.13 Symbol:
AIB Ready EAIBRDY
Address Offset: 00034h Default Value: 00000000h Access: Read Only Indicates if the corresponding AIB link is ready. Bits 31:6 5 4 3 2 1 0 Reserved. Flow Control Crossbar A1 AIB Ready. Flow Control Crossbar A0 AIB Ready. Flow Control Crossbar B1 AIB Ready. Flow Control Crossbar B0 AIB Ready. Scheduler 1 AIB Ready. Scheduler 0 AIB Ready. Description
3.4.2.14 Symbol:
Enable Port Processor EEN
Address Offset: 00038h Default Value: 00000000h Access: Read/Write Indicates that the Port Processor is enabled (1) or disabled (0). Bits 31:1 0 Reserved This bit indicates that the port processor is enabled (1) or disabled (0). The input port processor drops all cells (including Control Packets) from the linecard or crossbar and ignores all scheduler control information if this bit is 0. Description
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
177
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.15 Symbol:
AIB Enable Transmitter EAIBEN
Address Offset: 0003Ch Default Value: 00000000h Access: Read/Write Used to enable the transmitter of the corresponding AIB link. If the corresponding central Scheduler is not physically present in the system, this bit should be set to zero to reduce unwanted electrical noise and to minimize unwanted effects when either Scheduler board is inserted. Bits 31:6 5 4 3 2 1 0 Reserved Flow Control Crossbar A1 AIB Enable Tx. Flow Control Crossbar A0 AIB Enable Tx. Flow Control Crossbar B1 AIB Enable Tx. Flow Control Crossbar B0 AIB Enable Tx. Scheduler 1 AIB Enable Tx. Scheduler 0 AIB Enable Tx. Description
3.4.2.16 Symbol:
LCS Started/Stopped Status/Control ELCSSSS
Address Offset: 00040h Default Value: 00000000h Access: Read/Write This register resets to 0 (all subports stopped). Traffic must be started by Processed Control Packet or OOB after reset. Bits 31:8 7 6 5 4 3 2 1 Reserved Subport 3 egress MC queues are "lossy" if LCS Stopped Subport 2 egress MC queues are "lossy" if LCS Stopped Subport 1 egress MC queues are "lossy" if LCS Stopped Subport 0 egress MC queues are "lossy" if LCS Stopped Subport 3 is started if 1, stopped if 0 Subport 2 is started if 1, stopped if 0 Subport 1 is started if 1, stopped if 0 Description
178
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Bits 0
Description (Continued) Subport 0 is started if 1, stopped if 0 NOTE: The "lossy" behavior is defined in the "LCS Protocol Specification".
3.4.2.17 Symbol:
Lost Grant or Cell Body Debug Info ELGCBD
Address Offset: 00044h Default Value: 00000000h Access: Read and Clear Indicates the number of lost grants or cells. Bits 31:30 29:12 11:0 Reserved The first LCS grant label for which a cell was expected but lost. The total number of cells which have been expected but lost. Description
3.4.2.18 Symbol:
Scheduler Grant to Empty Queue Debug Info ESGEQD
Address Offset: 00048h Default Value: 00000000h Access: Read and Clear Indicates the number of grants to an empty queue. Bits 31:27 26:16 15:0 Reserved The first QID for which there was a scheduler grant to empty queue The total number of scheduler grants to empty queues Description
3.4.2.19 Symbol:
LCS Header CRC Error Count ELCSECT
Address Offset: 0004Ch Default Value: 00000000h Access: Read and Clear
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
179
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Indicates the number of CRC Errors in Linecard to Switch LCS headers Bits 31:0 Description The total number of detected CRC Errors in Linecard to Switch LCS headers
3.4.2.20 Symbol:
LCS Processor CP Subport 0 CRC Error Count ELCSP0CT
Address Offset: 00050h Default Value: 00000000h Access: Read and Clear Indicates the number of CRC Errors in processed CP from Subport 0. Bits 31:0 Description The total number of detected CRC Errors in Processed CPs from Subport 0.
3.4.2.21 Symbol:
LCS Processor CP Subport 1 CRC Error Count ELCSP1EC
Address Offset: 00054h Default Value: 00000000h Access: Read and Clear Indicates the number of CRC Errors in processed CP from Subport1. Bits 31:0 Description The total number of detected CRC Errors in Processed CPs from Subport 1.
3.4.2.22 Symbol:
LCS Processor CP Subport 2 CRC Error Count ELCSP2EC
Address Offset: 00058h Default Value: 00000000h Access: Read and Clear Indicates the number of CRC Errors in processed CP from Subport 2. Bits 31:0 Description The total number of detected CRC Errors in Processed CPs from Subport 2.
180
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.23 Symbol:
LCS Processor CP Subport 3 CRC Error Count ELCSP3EC
Address Offset: 0005Ch Default Value: 00000000h Access: Read and Clear Indicates the number of CRC Errors in processed CP from Subport 3. Bits 31:0 Description The total number of detected CRC Errors in Processed CPs from Subport 3.
3.4.2.24 Symbol:
Input Subport 0 Valid Count EIS0VCT
Address Offset: 00060h Default Value: 00000000h Access: Read and Clear Indicates the number of cells matched with grant sequence numbers Bits 31:0 Description The total number of cells matched with grant sequence numbers for Subport 0.
3.4.2.25 Symbol:
Input Subport 1 Valid Count EIS1VCT
Address Offset: 00064h Default Value: 00000000h Access: Read and Clear Indicates the number of cells matched with grant sequence numbers Bits 31:0 Description The total number of cells matched with grant sequence numbers for Subport 1.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
181
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.26 Symbol:
Input Subport 2 Valid Count EIS2VCT
Address Offset: 00068h Default Value: 00000000h Access: Read and Clear Indicates the number of cells matched with grant sequence numbers Bits 31:0 Description The total number of cells matched with grant sequence numbers for Subport 2.
3.4.2.27 Symbol:
Input Subport 3 Valid Count EIS3VCT
Address Offset: 0006Ch Default Value: 00000000h Access: Read and Clear Indicates the number of cells matched with grant sequence numbers Bits 31:0 Description The total number of cells matched with grant sequence numbers for Subport 3.
3.4.2.28 Symbol:
Output Subport 0 Valid Count EOS0VCT
Address Offset: 00070h Default Value: 00000000h Access: Read and Clear Indicates the number of cells matched with grant sequence numbers Bits 31:0 Description The total number of cells sent to Subport 0.
182
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.29 Symbol:
Output Subport 1 Valid Count EOS1VCT
Address Offset: 00074h Default Value: 00000000h Access: Read and Clear Indicates the number of cells matched with grant sequence numbers Bits 31:0 Description The total number of cells sent to Subport 1.
3.4.2.30 Symbol:
Output Subport 2 Valid Count EOS2VCT
Address Offset: 00078h Default Value: 00000000h Access: Read and Clear Indicates the number of cells matched with grant sequence numbers Bits 31:0 Description The total number of cells sent to Subport 2.
3.4.2.31 Symbol:
Output Subport 3 Valid Count EOS3VCT
Address Offset: 0007Ch Default Value: 00000000h Access: Read and Clear Indicates the number of cells matched with grant sequence numbers Bits 31:0 Description The total number of cells sent to Subport 3.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
183
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.32 Symbol:
Subport 0 to OOB FIFO Status ES0OOBS
Address Offset: 00080h Default Value: 00000000h Access: Read/Write Write of any value will dequeue the head of the FIFO. Bits 31:7 6:4 3:0 Reserved The offset of the head of the 8-deep FIFO in DS queue memory. The length (0 to 8) of the FIFO in DS queue memory. Description
3.4.2.33 Symbol:
Subport 1 to OOB FIFO Status ES1OOBS
Address Offset: 00084h Default Value: 00000000h Access: Read/Write Write of any value will dequeue the head of the FIFO. Bits 31:7 6:4 3:0 Reserved The offset of the head of the 8-deep FIFO in DS queue memory. The length (0 to 8) of the FIFO in DS queue memory. Description
184
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.34 Symbol:
Subport 2 to OOB FIFO Status ES2OOBS
Address Offset: 00088h Default Value: 00000000h Access: Read/Write Write of any value will dequeue the head of the FIFO. Bits 31:7 6:4 3:0 Reserved The offset of the head of the 8-deep FIFO in DS queue memory. The length (0 to 8) of the FIFO in DS queue memory. Description
3.4.2.35 Symbol:
Subport 3 to OOB FIFO Status ES3OOBS
Address Offset: 0008Ch Default Value: 00000000h Access: Read/Write Write of any value will dequeue the head of the FIFO. Bits 31:7 6:4 3:0 Reserved The offset of the head of the 8-deep FIFO in DS queue memory. The length (0 to 8) of the FIFO in DS queue memory. Description
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
185
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.36 Symbol:
Send OOB to Linecard Control Packet ESOBLCP
Address Offset: 00090h Default Value: 00000000h Access: Read/Write Before sending a CPU (OOB to linecard) control packet, the OOB must first write the control packet header and payload data into the appropriate locations in the Dataslice. See section 3.3, Table 28, "EPP Egress Control Packet Data Format and Dataslice OOB Addressing." The header for customer-specific CPU control packets should be written into the Dataslices as shown, but the payload data is completely up to the customer. To send the CPU control packet which has been written into Dataslice queue memory, the OOB writes the ESOBLCP register with the control packet type select in bits 5:4 (see the bit-breakout), and a linecard fanout in bits 3:0. If the port is connected to 4 subport linecards, then bits 3:0 are a subport-bitmap. If the port is connected to one OC-192-rate linecard, then bit 0 must be set when the OOB wishes to send a CP. When the CP has been sent to the linecard(s) indicated in bits 3:0, bits 3:0 will read back as 0. Since control packets have higher priority than any other traffic type, they will be sent immediately, unless the EPP is programmed to send only idle cells. Bits 31:6 5:4 3 2 1 0 Reserved Control Packet type select 0: Generic OOB to Subport Control Packet 2: LCS Stop Control Packet 3: LCS Start Control Packet Subport 3 fanout for OOB to Subport Control Packet Subport 2 fanout for OOB to Subport Control Packet Subport 1 fanout for OOB to Subport Control Packet Subport 0 fanout for OOB to Subport Control Packet Description
186
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.37 Symbol:
Suggested TDM Sync Subport Select ESTDMSSS
Address Offset: 00094h Default Value: 00000000h Access: Read/Write Sync Support Select Bits 31:2 1:0 Reserved Subport select. Subport from which (suggested) TDM Sync Processed Control packets will be accepted and forwarded to the scheduler, if this port is not enabled to internally generate suggested TDM Syncs. Description
3.4.2.38 Symbol:
Multicast Invalid Fanout Count EMCIFCT
Address Offset: 00098h Default Value: 00000000h Access: Read and Clear Fanout count Bits 31:30 29:28 27:16 15:0 Reserved First subport from which a multicast cell with all-0 fanout was received. First tag which yielded the all-0 fanout. Total number of cells with invalid fanouts that have been received (and dropped). Description
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
187
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.39 Symbol:
Internal Delay Matching Adjustments EIDMA
Address Offset: 0009Ch Default Value: 03A8D66Bh Access: Read/Write Delay matching Bits 31:29 28:27 26:23 Reserved DS iFIFO Token Mechanism Delay. Specifies the depth of the token shift register. Default at reset is 0x0. See Appendix C.2.2.1 for details. Flow Control Crossbar oEPP->iEPP Delay. Specifies the time it takes for a Flow Control frame to reach the iEPP after it sent from any oEPP. The value of this register will not require adjustment unless there is a change in timing in a future revision of the EPP or Crossbar. Unicast Scheduler Request to Waiting Delay. Specifies the minimum delay for a priority 0 UC scheduler request to be granted. The value of this register will not require adjustment unless there is a change in timing in a future revision of the EPP or Scheduler. TDM Ingress Subport to Scheduler Grant Delay. Specifies the delay between a valid input TDM slot and the corresponding scheduler TDM grant. The value of this register will not require adjustment unless there is a change in timing in a future revision of the EPP or Scheduler. TDM Egress Fanout Delay. Specifies the delay between a valid output TDM slot and the arrival of the TDM cell. The value of this register will not require adjustment unless there is a change in timing in a future revision of the EPP, DS, Crossbar or Scheduler. LCS Unicast Grant to Scheduler Request Delay. The value of this register will not require adjustment, provided that the customer can meet the RTT constraint given in Section 1.1 "Preventing Underflow of the VOQ" on page 327. Description
22:18
17:12
11:6
5:0
188
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.40 Symbol:
Output Backpressure/Unbackpressure Threshold EOPBPTHR
Address Offset: 000A0h Default Value: 00001C46h Access: Read/Write These thresholds are for Multicast queues only. Bits 31:15 Reserved Output Unbackpressure Threshold. This specifies when the output port processor de-asserts backpressure. When the Output Scheduler grants to an output queue, the output port processor checks if the number of cells in that queue is equal to the threshold value. If the two values are equal, then an unbackpressure signal is sent to the central scheduler. If the EPP is in OC-48c mode, then this value should be changed to 1Fh. Reserved Output Backpressure Threshold. This specifies when the output port processor asserts backpressure. When a cell from crossbar enters an output queue, the output port processor checks if the number of cells already in that queue is equal to the threshold value. If the two values are equal, then a backpressure signal is sent to the central scheduler. Description
14:8
7 6:0
3.4.2.41 Symbol:
Egress OC-48/OC-192c Mode Map EEMM
Address Offset: 000A4h Default Value: 00000000h Access: Read/Write Chooses OC-48c/OC-192c Mode. Bits 31:0 Description Port Mode select. Bits 0:31correspond to individual ports. If the bit is set to 1, the port is OC-48; if set to 0, the port is OC-192c.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
189
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.42 Symbol:
TDM Offset Value ETDMO
Address Offset: 000A8h Default Value: 00000080h Access: Read/Write Determines TDM offset value. Bits 31:8 7:0 Reserved Specifies TDM Offset Value. The number of cell times from the arrival of TDM sync from the central scheduler to the recognition of the first time slot in the TDM table. Note that this value must be identical in all ports participating in TDM traffic. Description
3.4.2.43 Symbol:
Freeze Control EFCTL
Address Offset: 000ACh Default Value: 00000000h Access: Read/Write Freezes TDM grants, requests and cells. Bits 31:6 5 4 3 2 1 0 Reserved Freeze TDM grants from LCS Grant Manager. If this bit is 1, the LCS Grant Manager sends no TDM grants to the linecard. Freeze TDM Scheduler Requests. If this bit is 1, TDM input requests are not sent to the Scheduler. Freeze TDM cells from Output Scheduler. If this bit is 1, the Output Scheduler does not schedule any TDM cells onto the output line. Freeze MC/UC grants from LCS Grant Manager. If this bit is 1, the LCS Grant manager sends no MC or UC grants to the linecard. Freeze Scheduler Request Modulator. If this bit is 1, the SRM sends no MC or UC requests to the scheduler. Freeze MC/UC cells from Output Scheduler. If this bit is 1, the Output Scheduler does not schedule any MC or UC cells onto the output line. Description
190
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.44 Symbol:
Freeze Unicast Output Scheduler Flows EFUOS
Address Offset: 000B0h Default Value: 00000000h Access: Read/Write Determines whether or not the Output Scheduler sends unicast cells from specific ports. Bits 31:0 Description Bits 31:0 correspond to specific ports. If the bit for the port is set to 1, the Output Scheduler sends no unicast cells from that port.
3.4.2.45 Symbol:
Multicast Priority Drop EMPD
Address Offset: 000B4h Default Value: 00000000h Access: Read/Write This specifies if a given priority should allow multicast cells to be dropped. Bits 31:4 Reserved This specifies if a given priority should allow multicast cells to be dropped. If a bit is 1, then the output port processor does not assert backpressure for the corresponding multicast queue. If a bit is 0, then the output port processor prevents dropping of multicast cells by asserting backpressure to the central scheduler. Bit 3 is for priority 3, bit 2 is for priority 2, bit 1 is for priority 1, and bit 0 is for priority 0. Description
3:0
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
191
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.46 Symbol:
Fault Tolerance Status/Control EFTCTRL
Address Offset: 000B8h Default Value: 00000000h Access: Read/Write Fault Tolerence Status/Control Register. Bits 31:5 Reserved Primary Flow Control Crossbar B Select. This bit specifies which Flow Control Crossbar B is primary. If this bit is 0, then Flow Control Crossbar B 0 is primary. If this bit is 1, then Flow Control Crossbar B 1 is primary. This bit is both a status and control bit. When the primary Flow Control Crossbar B goes down, the EPP must switch over to the secondary Flow Control Crossbar B (causes this bit to flip). OOB can also change the configuration. Primary Flow Control Crossbar A Select. This bit specifies which Flow Control Crossbar A is primary. If this bit is 0, then Flow Control Crossbar A 0 is primary. If this bit is 1, then Flow Control Crossbar A 1 is primary. This bit is both a status and control bit. When the primary Flow Control Crossbar A goes down, the EPP must switch over to the secondary Flow Control Crossbar A (causes this bit to flip). OOB can also change the configuration. Ignore Central Scheduler Grants. This bit indicates that the EPP is ignoring grants from both primary and secondary central schedulers. The EPP asserts this bit if both schedulers have transmitted CRC errors in the same frame, if the only operational scheduler has transmitted a CRC error, or if the only operational scheduler's AIB link dropped ready. Note that the OOB must write a 0 to this bit to clear it. Freeze Scheduler Request Modulator (Multicast & Unicast only). This bit indicates that the SRM is frozen (TDM traffic can still flow) due to fault tolerance errors. The EPP asserts this bit if both schedulers have transmitted CRC errors in the same frame, if the only operational scheduler has transmitted a CRC error, or if the only operational scheduler's AIB link dropped ready. Note that the OOB must write a 0 to this bit to clear it. Primary Scheduler Select. This bit specifies which scheduler is primary. If this bit is 0, then scheduler 0 is primary. If this bit is 1, then scheduler 1 is primary. This value must be consistent on all the port processors. This bit is both a status and control bit. When the primary scheduler goes down, all EPPs must switch over to the secondary scheduler (causes this bit to flip). The OOB processor can also change the configuration. Description
4
3
2
1
0
192
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.47 Symbol:
Refresh Schedulers Command ERFRSSC
Address Offset: 000BCh Default Value: 00000000h Access: Write Only .Causes the EPP to start the refresh operation. Bits Description Start Refresh Schedulers Operation. A write to this address (any data) causes the EPP to start the scheduler refresh operation. The EPP freezes the Scheduler Request Modulator, freezes the Output Scheduler, and waits for in-flight cells to arrive before transferring state information to the scheduler(s). Note that OOB must must first disable the scheduler's non-TDM traffic register for this port before issuing this command.
31:0
3.4.2.48 Symbol:
Go Schedulers Command ESTRTSC
Address Offset: 000C0h Default Value: 00000000h Access: Write Only A write to this address causes an EPP to end the refresh operation. Bits Description Synchronously Start Schedulers Operation. A write to this address (any data) causes the EPP to end the refresh operation. Since the primary and secondary scheduler must transmit the same control in lock step, this command is used to synchronize the two schedulers. Note that OOB must first enable the scheduler's non-TDM traffic register for this port before issuing this command.
31:0
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
193
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.49 Symbol:
Idle Count Value EIDLCT
Address Offset: 000C4h Default Value: 00000000h Access: Read/Write
This register is used to compensate for the clock frequency variation between the ETT1 switch and the linecard.
Bits
Description Idle Count Value. This represents the frequency of sending idle frames to the line card. This register is used to compensate for the clock frequency variation between the ETT1 switch and the linecard. If this register is set to 0x0, only idle frames are transmitted to the linecard. If this register is set to 0xFFFFFFFF, no idle frames are transmitted. If this register is set to N, an idle frame is sent every N + 1 frame times. Note that this register resets to 0x0.
31:0
3.4.2.50 Symbol:
Enable TDM Sync Generation ETDMEN
Address Offset: 000C8h Default Value: 00000000h Access: Read/Write Indicates if the TDM sync generation is enabled (1) or disabled (0). Bits 31:1 0 Reserved. Enable TDM Sync Generation. This bit indicates if the TDM sync generation is enabled (1) or disabled (0). If the linecard chooses not to provide the TDM sync signal, any one of the EPPs can be programmed to generate this signal. Description
194
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.51 Symbol:
TDM Sync Generation Control ETDMCTRL
Address Offset: 000CCh Default Value: 00000000h Access: Read/Write TDM Sync Generation Control register. Bits 31:13 12 11:0 Reserved. TDM Frame Select. This bit specifies which TDM frame table to select. TDM Sync Frequency. This represents the frequency of the TDM sync pulse. If this register is set to N, a TDM sync signal is sent to the central Scheduler every N + 1 frame times. Description
3.4.2.52 Symbol:
No Grant Count ENOGCT
Address Offset: 000DOh Default Value: FFFFFFFFh Access: Read/Write No Grant Count register. Bits Description Controls the frequency of sending cells without grants to the line card. This register is used to guarantee that the linecard gets a gap in the stream of grants often enough so that it can send idle cells without holding up cell payloads. If this register is set to 0x0, no grants are sent to the linecard. If this register is set to 0xFFFFFFFF, no grants are held up. If this register is set to N, a cell without a grant is sent every N+1 non-idle frame times. Note that this register resets to 0xFFFFFFFF. In OC-48 mode, (N+1) should be odd so that no-grant celltimes are distributed equally to all four linecards, so the value written to this register (N) should be even.
31:0
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
195
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.53 Symbol:
End of Sch Refresh/Sch OC-48 Sync ModCount EESR48SD
Address Offset: 000D4h Default Value: 00000000h Access: Write Only Specifies the value of an internal OC-48 Sync counter at which the OOB Go Schedulers command should trigger the end of Scheduler refresh. Bits 31:5 4:0 Reserved. Prevents OC-48 Sync phase change after Scheduler refresh. The value of this register will not require adjustment unless there is a change in timing in a future revision of the EPP or Scheduler. Description
3.4.2.54 Symbol:
PLL Status/Control EPLLREG
Address Offset: 00100h Default Value: 0001447Ch Access: Read/Write For more details, refer to the IBM SA-12E databook. Bits 31:21 30:20 19:17 16 15:10 9:6 5:3 2:0 Reserved. PLL_OBSERVE (read only). Reserved PLL_RESET. When set, VCO is held at minimum frequency and PLL is bypassed. PLL reset takes 10mS to complete. PLL_TUNE. PLL_MULT. PLL_RANGE_B. PLL_RANGE_A. Description
196
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.55 Symbol:
TDM Frame Table 0 Memory ETDMFT0M
Address Offset: 02000-02FFCh Default Value: 000000000h Access: Read/Write Each entry represents a time slot in the TDM table. Bits 31:23 22 21:20 19:10 9 8:4 3:0 Reserved. Input Valid. This bit is 1 if the input side TDM entry is valid. Input Subport. (OC-48 only) Input linecard which provides the TDM cell to send. Input Tag . The tag that the head of the input TDM tag FIFO must match. Output Valid. This bit is 1 if the output side TDM entry is valid. Input Port. The input port from which the output expects to receive a TDM cell. Egress Subport Fanout. Specifies which linecard(s) will receive the TDM cell. Description
3.4.2.56 Symbol:
TDM Frame Table 1 Memory ETDMFT1M
Address Offset: 03000-03FFCh Default Value: 000000000h Access: Read/Write Each entry represents a time slot in the TDM table. Bits 31:23 22 21:20 19:10 9 8:4 3:0 Reserved. Input Valid. This bit is 1 if the input side TDM entry is valid. Input Subport. (OC-48 only) Input linecard which provides the TDM cell to send. Input Tag. The tag that the head of the input TDM tag FIFO must match. Output Valid. This bit is 1 if the output side TDM entry is valid. Input Port. The input port from which the output expects to receive a TDM cell. Egress Subport Fanout. Specifies which linecard(s) will receive the TDM cell. Description
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
197
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.57 Symbol:
ITIB Memory EITIBM
Address Offset: 04000-07FFCh Default Value: 000000000h Access: Read/Write ITIB memory. Bits 31:0 Description ITIB Memory. Each entry is a multicast port fanout value, addressed by a12-bit multicast tag. Note that an all-zero fanout is considered invalid.
3.4.2.58 Symbol:
Linecard Request Count Memory ESRCTM
Address Offset: 08000-0FFFCh Default Value: 000000000h Access: Read/Write This memory is addressed by {source_subport, QID}. Each traffic type has a different address and data format. For Unicast Traffic: Each request count is 10 bits wide if OC-192c (9:0), 8 bits wide if OC-48(7:0). Address Bits 19:15 14:13 12:11 10:9 8:4 3:2 1:0 Data Bits 31:10 9:0 Reserved. Request Count Description Must be set to 0x1 Source Subport Must be set to 0x2 for Unicast Traffic type Priority Destination Port Destination Subport Must be set to 0 Description
198
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
For Multicast traffic: Each request count is 10 bits wide if OC-192c, 8 bits wide if OC-48. Each request count corresponds to a FIFO in Linecard Multicast Tag FIFO Memory; if the request count is changed by OOB, the FIFO length also changes. OOB should specify a new head address for the FIFO in the upper data bits. OOB should also write the correct number of tags into the Linecard Multicast Tag FIFO memory so that the state of the request counter and its tag FIFO are consistent. Address Bits 19:15 14:13 12:11 10:9 8:0 Data Bits 31:20 19:10 9:0 Reserved. Linecard Multicast Tag Head Request Count Description Must be set to 0x1 Source Subport Must be set to 0x3 for Multicast Traffic type Priority Must be set to 0 Description
For TDM traffic: Each request count is 10 bits wide if OC-192c, 8 bits wide if OC-48. Each request count corresponds to a FIFO in Linecard TDM Tag FIFO Memory; if the request count is changed by OOB, the FIFO length also changes. OOB should specify a new head address for the FIFO in the upper data bits. OOB should also write the correct number of tags into the Linecard TDM Tag FIFO memory so that the state of the request counter and its tag FIFO are consistent. Address Bits 19:15 14:13 12:9 8:4 3:2 1:0 Must be set to 0x1 Source Subport Must be set to 0x7 for TDM Traffic type Must be set to 0 Destination Subport Must be set to 0 Description
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
199
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Data Bits 31:20 19:10 9:0 Reserved. Linecard TDM Tag Head Request Count Description
For Control Packet: Each request count is 3 bits wide. Each request count corresponds to a 7-deep, 1-bit-wide FIFO. Whenever OOB changes a request count value, it should also supply any tags associated with requests in the upper data bits. The LSB of the "Control Packet Tag FIFO" field is the head of the tag FIFO, i.e. the next tag that will be sent with a grant. Address Bits 19:15 14:13 12:9 8:4 3:2 1:0 Data Bits 31:10 9:3 2:0 Reserved. Control Packet Tag Head Request Count Description Must be set to 0x1 Source Subport Must be set to 0x0 for Control Packet Traffic type Must be set to 0 Destination Subport Must be set to 0 Description
3.4.2.59 Symbol:
Input Queue Information Memory EIQIM
Address Offset: 16000-17FFCh Default Value: 000000000h Access: Read/Write This memory is addressed by QID. This memory contains input queue information for 4 TDM, 4 multicast, 128 unicast, and 4 control packet queues. For Unicast Traffic: 128-512 queues depending on Egress OC48 Mode Map register; each OC-192 queue holds 0-64 cells, each OC-48 queue holds 0-16 cells.
200
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Address Bits 19:13 12:11 10:9 8:4 3:2 1:0 Data Bits 31:13 12:7 6:0 Reserved. Head Length Description Must be set to 0xB Must be set to 0x2 for Unicast Traffic type Priority Destination Port Destination Subport Must be set to 0 Description
For Multicast traffic: 4 queues regardless of OC-48 modes; each queue holds 0-96 cells. Address Bits 19:13 12:11 10:9 8:0 Data Bits 31:15 14 13:7 6:0 Reserved. Full Head Tail Description Must be set to 0xB Must be set to 0x3 for Multicast Traffic type Priority Must be set to 0 Description
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
201
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
For TDM traffic: 1 or 4 queues depending on this EPP's OC-48 mode. If OC-192, up to 96 cells; if OC-48, up to 24 cells each queue. Address Bits 19:13 12:9 8:4 3:2 1:0 Data Bits 31:15 14 13:7 6:0 Reserved. Full Head Tail Description Must be set to 0xB Must be set to 0x7 for TDM Traffic type Must be set to 0 Source Subport Must be set to 0 Description
For Control Packet: 1 or 4 queues depending on this EPP's OC-48 mode. 8 cells per queue regardless of mode Address Bits 19:13 12:9 8:4 3:2 1:0 Data Bits 31:7 6:4 3:0 Reserved. Head Length Description Must be set to 0xB Must be set to 0x0 for Control Packet Traffic type Must be set to 0 Source Subport Must be set to 0 Description
202
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.60 Symbol:
Outstanding Scheduler Request Count Memory EOSRM
Address Offset: 1A000-1BFFCh Default Value: Unknown Access: Read/Write The Outstanding Scheduler Request Count Memory contains the number of cells in a VOQ that the EPP must request a grant from the Scheduler. This memory is addressed by QID, but only Unicast and Multicast addresses are valid. For Unicast traffic: 128-512 queues depending on Egress OC-48c Mode Map register; maximum 64 requests for OC-192c, max 16 requests for OC-48. Address Bits 19:13 12:11 10:9 8:4 3:2 1:0 Data Bits 31:7 6:0 Reserved. Requests Description Must be set to 0xD Must be set to 0x2 for Unicast traffic type Priority Destination Port Destination Subport Must be set to 0 Description
For Multicast traffic: 4 queues regardless of OC-48 modes; max 96 requests. Address Bits 19:13 12:11 10:9 8:0 Must be set to 0xD Must be set to 0x3 for Multicast traffic type Priority Must be set to 0 Description
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
203
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Data Bits 31:7 6:0 Reserved. Requests Description
3.4.2.61 Symbol:
Waiting Scheduler Request Count Memory EWSRCT
Address Offset: 1C000-1DFFCh Default Value: Unknown Access: Read/Write The Waiting Scheduler request Count Memory contains the number of requests that the EPP is waiting to get a grant back from the Scheduler. This memory is addressed by QID, but only Unicast and Multicast addresses are valid. After resetting an EPP, write 00000000h to address offsets 1D000h through 1D7FCh in that EPP (a total of 512 OOB writes). Whan all EPPs are reset simultaneously (using a broadcast OOB write to the EPP Reset and Control register), 512 broadcast OOB writes can be used to reset those locations in all EPPs. For Unicast traffic: 128-512 queues depending on Egress OC-48c Mode Map register; maximum 64 requests for OC-192c, max 16 requests for OC-48. Address Bits 19:13 12:11 10:9 8:4 3:2 1:0 Data Bits 31:7 6:0 Reserved. Requests Description Must be set to 0xE Must be set to 0x2 for Unicast traffic type Priority Destination Port Destination Subport Must be set to 0 Description
204
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
For Multicast traffic: 4 queues regardless of OC-48 modes; max 96 requests. Address Bits 19:13 12:11 10:9 8:0 Data Bits 31:7 6:0 Reserved. Requests Description Must be set to 0xE Must be set to 0x3 for Multicast traffic type Priority Must be set to 0 Description
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
205
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.4.2.62 Symbol:
Output UC Queue Debit Count Memory EOUCQDCT
Address Offset: 1E000-1E7FCh Default Value: 000000000h Access: Read/Write This memory contains debits for UC only. Address Bits 19:13 12:11 10:9 8:4 3:2 1:0 Must be set to 0xF Must be set to 0x0 for Unicast traffic type Priority Destination Port Destination Subport Must be set to 0 Description
Data Bits 31:7 6:0 Reserved. Debits. 128-512 queues depending on Egress OC-48c Mode Map register; maximum 64 debits for OC-192c, max 16 debits for OC-48. Description
3.4.2.63 Symbol:
Output UC Queue Information Memory EOUCQI
Address Offset: 20000-21FFCh Default Value: 000000000h Access: Read/Write This memory is addressed by QID, but only UC qid addresses are valid. 128-512 queues depending on Egress OC-48c Mode Map register; each OC-192c queue holds 0-64 cells, each OC-48 queue holds 0-16 cells.
206
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Address Bits 19:13 12:11 10:9 8:4 3:2 1:0 Data Bits 31:13 12:7 6:0 Reserved. Head Length Description Must be set to 0x10 Must be set to 0x2 for Unicast traffic type Priority Destination Port Destination Subport Must be set to 0 Description
3.4.2.64 Symbol:
Output MC Queue Drop Counters EOMCQDCT
Address Offset: 3E000-3FFFCh Default Value: 000000000h Access: Read/Write This set of registers is addressed by QID, but only MC addresses are valid. Each register contains the number of MC cells dropped by the output EPP for this QID. Note that this value will stick at 0xFFFFFFFF. There is only one register for each MC output queue (total of 4 registers, regardless of OC-48 mode). Address Bits 19:13 12:11 10:9 8:2 1:0 Must be set to 0x1F Must be set to 0x3 for Multicast traffic type Priority Must be set to 0 Must be set to 0 Description
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
207
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Data Bits 31:0 Drop Count. Description
3.4.2.65 Symbol:
OTIB Memory EOTIBM
Address Offset: 08000-FFFFCh Default Value: 000000000h Access: Read/Write OTIB memory. Each entry is a multicast egress subport fanout value, addressed by {multicast_tag, source_port}. Note that if the egress port is OC-48, an all-0 fanout will cause the cell to be dropped. If the egress port is OC-192c, this memory is not used. Bits 31:4 3:0 Reserved. Fanout Description
208
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.5
ENHANCED PORT PROCESSOR SIGNAL DESCRIPTIONS
This section describes the Enhanced Port Processor signals. The following notation is used to describe the signal type: I O B Input pin Output pin Bidirectional Input/Output pin
The signal description also includes the type of buffer used for the particular signal: PECL STI PECL are Pseudo-ECL (positive voltage ECL) compatible signals. STI is a very high-speed asynchronous communications protocol used to connect devices that may be 1 to 15 meters apart. All HSTL specifications are consistent with EIA/JEDEC Standard, EIA/JESD8-6 High Speed Transceivers Logic (HSTL): A 1.5V Output Buffer Supply Voltage based Interface Standard for Digital Integrated Circuits", dated 8/95. Refer to these specifications for more information. The CMOS Buffers are either 3.3 V compatible or 2.5 V Low Voltage TTL compatible signals, as noted.
Table 25. Enhanced Port Processor Signal Descriptions
HSTL
CMOS
Name
I/O
Type OOB Interface
Description
oob_ad[7:0] oob_clk oob_int_hi oob_int_lo oob_valid_L oob_wait_L oob_devsel0 oob_devsel1 oob_devsel2
B I O O I O I I I
CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS
OOB Address bus OOB Clock OOB Interrupt OOB Interrupt OOB Control OOB Control OOB Device Select OOB Device Select OOB Device Select
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
209
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 25.
Enhanced Port Processor Signal Descriptions (Continued)
Name oob_devsel3
I/O I
Type CMOS
Description OOB Device Select
EPP/DS Interface d2p_d0_id[7:0] d2p_d0_od[7:0] d2p_d1_id[7:0] d2p_d1_od[7:0] d2p_d2_id[7:0] p2d_d[6:0]_ic[7:0] p2d_d[6:0]_oc[7:0] p2d_d0_id[7:0] p2d_d1_id[7:0] Vref I I I I I O O O O I HSTL Class 1 HSTL Class 1 HSTL Class 1 HSTL Class 1 HSTL Class 1 HSTL Class 1 HSTL Class 1 HSTL Class 1 HSTL Class 1 HSTL DS to EPP ingress data unmodified DS to EPP egress data DS to EPP ingress data unmodified DS to EPP egress data DS to EPP ingress data unmodified EPP to DS ingress control EPP to DS egress control EPP to DS ingress data modified EPP to DS ingress data modified Input Reference Voltage
AIB Interface p2s_s0_[c cn, e, en, o, on] p2s_s1_[c cn, e, en, o, on] s2p_s0_[c cn, e, en, o, on] s2p_s1_[c cn, e, en, o, on] f2p_a0_[c cn, e, en, o, on] O O I I I STI STI STI STI STI EPP to Sched AIB EPP to Sched AIB Sched to EPP AIB Sched to EPP AIB Fc Crossbar to EPP AIB
210
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 25.
Enhanced Port Processor Signal Descriptions (Continued)
Name f2p_a1_[c cn, e, en, o, on] f2p_b0_[c cn, e, en, o, on] f2p_b1_[c cn, e, en, o, on] p2f_a0_[c cn, e, en, o, on] p2f_a1_[c cn, e, en, o, on] p2f_b0_[c cn, e, en, o, on] p2f_b1_[c cn, e, en, o, on]
I/O I I I I I I I
Type STI STI STI STI STI STI STI
Description Fc Crossbar to EPP AIB Fc Crossbar to EPP AIB Fc Crossbar to EPP AIB EPP to Fc Crossbar AIB EPP to Fc Crossbar AIB EPP to Fc Crossbar AIB EPP to Fc Crossbar AIB
Power Supply, Clock Source, Reset, and Diagnostics soc_out VDDA VDD VDDQ ref_clk ref_clkn soc_in soc_inn plllock pwrup_reset_in_L I I I I O I O CMOS Isolated Supply Supply Supply PECL PECL PECL PECL CMOS CMOS Buffered soc_in (NC) VDD (2.5 V) for PLL VDD (2.5 V) VDDQ (1.6 V) System 200MHz clock System 200MHz clock System Start-of-cell System Start-of-cell Test output Synchronous, Active Low Reset
JTAG Interface jtag_tck jtag_tdi I I CMOS CMOS 1149.1 JTAG 2.5V ONLY! 1149.1 JTAG 2.5V ONLY!
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
211
PRELIMINARY Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 25.
Enhanced Port Processor Signal Descriptions (Continued)
Name jtag_tdo jtag_tms jtag_trst_L
I/O O I I
Type CMOS CMOS CMOS
Description 1149.1 JTAG 2.5V ONLY! 1149.1 JTAG 2.5V ONLY! 1149.1 JTAG 2.5V ONLY!
ASIC Manufacturing Test Interface ce0_io ce0_scan ce0_test ce0_tstm3 lssd_ce1_a lssd_ce1_b lssd_ce1_c1 lssd_ce1_c2 lssd_ce1_c3 lssd_scan_in[19:0] lssd_scan_out[19:0] plltest_in plltest_out test_di1 I I I I I I I I I I O I O I CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS Test (GND) Test (GND) Test (GND) Test (GND) Test (VDD) Test (VDD) Test (VDD) Test (VDD) Test (VDD) Scan input (GND) Scan output Test (GND) Test output (NC) Test (VDD) Should be driven to GND during reset. All outputs are tristated when low. Test (VDD) Should be driven to GND during reset. All outputs are tristated when low. Test (VDD) Test (GND) Test (VDD)
test_di2 test_lt test_re test_ri
I I I I
CMOS CMOS CMOS CMOS
212
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
213
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.6 3.6.1
PINOUT AND PACKAGE INFORMATION Pinout Tables
Table 26. Enhanced Port Processor Pinout (left side)
01 A B C D E F G H J K L M N P R T U V W Y AA AB AC AD AE
No Pin UNUSED lssd_scan_out18 UNUSED lssd_scan_out17 UNUSED lssd_scan_out16 d2p_d1_od4 d2p_d0_od2 d2p_d0_od1 d2p_d0_od7 p2d_d6_oc5 p2d_d3_oc0 p2d_d3_oc7 p2d_d2_oc5 p2d_d1_oc3 p2d_d1_oc2 p2d_d0_oc0 lssd_scan_in13 p2d_d1_oc5 lssd_scan_in12 p2d_d0_oc7 lssd_scan_in11 d2p_d2_id6 d2p_d2_id7
02
UNUSED GND UNUSED VDDQ UNUSED GND d2p_d0_od0 VDD d2p_d1_od3 GND UNUSED VDD p2d_d5_oc3 VDDQ p2d_d2_oc4 GND p2d_d0_oc1 VDD p2d_d1_oc4 GND p2d_d0_oc6 VDDQ d2p_d2_id5 GND d2p_d1_id0
03
test_di2 UNUSED lssd_scan_in18 UNUSED lssd_scan_in17 d2p_d1_od1 lssd_scan_in16 d2p_d0_od5 p2d_d6_oc3 p2d_d6_oc4 p2d_d5_oc2 p2d_d6_oc6 p2d_d5_oc4 p2d_d3_oc6 p2d_d3_oc1 UNUSED p2d_d2_oc0 p2d_d2_oc7 lssd_scan_out13 p2d_d0_oc3 lssd_scan_out12 d2p_d2_id4 lssd_scan_in10 d2p_d1_id1 lssd_scan_out10
04
UNUSED VDD UNUSED GND UNUSED VDD d2p_d1_od2 GND d2p_d0_od6 VDDQ p2d_d5_oc1 GND UNUSED GND p2d_d3_oc2 VDDQ p2d_d2_oc6 GND p2d_d0_oc2 VDD d2p_d2_id1 GND d2p_d1_id2 VDD d2p_d1_id7
05
ce0_test UNUSED ce0_scan UNUSED test_di1 UNUSED UNUSED d2p_d0_od4 UNUSED p2d_d5_oc6 p2d_d4_oc0 lssd_scan_in15 p2d_d5_oc7 lssd_scan_out14 p2d_d4_oc3 p2d_d4_oc6 p2d_d3_oc4 p2d_d1_oc1 p2d_d1_oc7 d2p_d2_id0 lssd_scan_out11 d2p_d1_id3 lssd_scan_in9 d2p_d0_id1 lssd_scan_out9
06
UNUSED GND UNUSED VDDQ UNUSED GND UNUSED VDDQ d2p_d0_od3 GND p2d_d4_oc1 VDD p2d_d4_oc5 VDD p2d_d4_oc2 GND p2d_d2_oc2 VDDQ d2p_d2_id2 GND d2p_d1_id6 VDDQ d2p_d0_id4 GND p2d_d1_id2
07
ce0_io UNUSED ce0_tstm3 UNUSED UNUSED UNUSED UNUSED d2p_d1_od0 d2p_d1_od5 p2d_d6_oc0 p2d_d5_oc5 lssd_scan_out15 p2d_d4_oc4 lssd_scan_in14 p2d_d4_oc7 p2d_d2_oc3 p2d_d1_oc0 p2d_d0_oc4 d2p_d2_id3 d2p_d1_id4 d2p_d0_id0 d2p_d0_id5 lssd_scan_in8 p2d_d1_id3 lssd_scan_out8
08
UNUSED VDDQ s2p_s0_o GND UNUSED VDD UNUSED GND UNUSED VDDQ p2d_d6_oc7 GND Vref GND p2d_d3_oc5 VDDQ p2d_d0_oc5 GND d2p_d1_id5 VDD UNUSED GND p2d_d0_id0 VDDQ d2p_d0_id7
214
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 27. Enhanced Port Processor Pinout (center)
09 A B C D E F G H J K L M N P R T U V W Y AA AB AC AD AE
UNUSED UNUSED s2p_s0_c s2p_s0_on s2p_s0_e s2p_s0_en UNUSED UNUSED Vref d2p_d1_od6 p2d_d6_oc1 d2p_d1_od7 p2d_d6_oc2 p2d_d3_oc3 Vref UNUSED p2d_d1_id1 d2p_d0_id3 p2d_d1_id0 p2d_d1_id7 p2d_d1_id6 p2d_d0_id1 p2d_d0_id7 d2p_d0_id6 p2d_d1_id5
10
UNUSED GND s2p_s0_cn VDD UNUSED GND UNUSED VDDQ Vref GND Vref VDD p2d_d5_oc0 VDD p2d_d1_oc6 GND d2p_d0_id2 VDDQ p2d_d0_id4 GND p2d_d6_ic2 VDD UNUSED GND p2d_d1_id4
11
UNUSED UNUSED UNUSED UNUSED p2s_s0_e p2s_s0_en UNUSED UNUSED UNUSED UNUSED VDD Vref GND p2d_d2_oc1 VDD Vref p2d_d0_id5 p2d_d6_ic3 p2d_d5_ic1 p2d_d5_ic5 p2d_d5_ic4 p2d_d6_ic5 p2d_d6_ic6 p2d_d0_id3 p2d_d0_id2
12
UNUSED VDDQ UNUSED GND lssd_scan_ out19 VDDQ lssd_scan_in19 GND UNUSED VDD UNUSED GND VDD GND p2d_d0_id6 VDD p2d_d5_ic0 GND lssd_scan_ out7 VDDQ lssd_scan_in7 GND p2d_d6_ic1 VDDQ p2d_d6_ic0
13
s2p_s1_en p2s_s0_c p2s_s0_cn s2p_s1_e test_lt UNUSED UNUSED test_re s2p_s1_o UNUSED GND VDD GND VDD GND p2d_d4_ic1 p2d_d6_ic4 Vref p2d_d5_ic3 p2d_d5_ic2 p2d_d4_ic0 UNUSED p2d_d4_ic4 p2d_d4_ic5 p2d_d6_ic7
14
UNUSED VDD UNUSED GND test_ri VDDQ jtag_trst_L GND UNUSED VDD UNUSED GND VDD GND p2d_d3_ic6 VDD p2d_d4_ic3 GND lssd_scan_in6 VDDQ lssd_scan_ out6 GND p2d_d3_ic2 VDD p2d_d3_ic3
15
UNUSED UNUSED UNUSED UNUSED p2s_s0_on p2s_s0_o UNUSED s2p_s1_on p2s_s1_c UNUSED VDD oob_devsel0 GND UNUSED VDD UNUSED p2d_d3_ic7 p2d_d3_ic0 p2d_d4_ic2 p2d_d5_ic6 p2d_d5_ic7 p2d_d4_ic7 p2d_d4_ic6 p2d_d2_ic0 p2d_d2_ic1
16
p2s_s1_on GND s2p_s1_c VDD UNUSED GND p2s_s1_cn VDDQ p2f_a0_cn GND oob_valid_L VDD UNUSED VDD Vref GND p2d_d0_ic1 VDDQ UNUSED GND p2d_d3_ic1 VDD p2d_d3_ic4 GND p2d_d2_ic7
17
p2s_s1_o f2p_a0_on s2p_s1_cn p2s_s1_e UNUSED UNUSED f2p_a0_en p2f_a0_c f2p_a0_e oob_ad1 oob_devsel3 UNUSED UNUSED UNUSED p2d_d1_ic1 Vref Vref p2d_d0_ic0 p2d_d1_ic2 p2d_d2_ic4 p2d_d2_ic5 p2d_d2_ic2 p2d_d3_ic5 p2d_d1_ic5 p2d_d2_ic6
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
215
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 28. Enhanced Port Processor Pinout (right side)
18
f2p_a0_o VDDQ p2s_s1_en GND UNUSED VDD p2f_a0_e GND f2p_a1_e VDDQ UNUSED GND UNUSED GND UNUSED VDDQ Vref GND p2d_d0_ic6 VDD p2d_d1_ic3 GND p2d_d2_ic3 VDDQ p2d_d1_ic4
19
jtag_tms UNUSED jtag_tck f2p_a0_c UNUSED p2f_a0_en oob_ad5 f2p_a1_en p2f_a1_cn p2f_a1_o f2p_b0_c lssd_ce1_c1 UNUSED UNUSED p2f_b0_on f2p_b1_cn p2f_b1_e UNUSED UNUSED p2d_d0_ic7 p2d_d0_ic3 p2d_d1_ic6 lssd_scan_out5 UNUSED lssd_scan_in5
20
UNUSED GND f2p_a0_cn VDDQ soc_out GND oob_ad4 VDDQ p2f_a1_on GND p2f_b0_cn VDD UNUSED VDD p2f_b0_e GND f2p_b1_c VDDQ UNUSED GND p2d_d0_ic5 VDDQ p2d_d1_ic7 GND p2d_d1_ic0
21
ref_clkn UNUSED ref_clk soc_in jtag_tdi UNUSED oob_ad0 p2f_a1_c UNUSED f2p_b0_cn p2f_b0_c lssd_ce1_c2 UNUSED lssd_ce1_c3 p2f_b0_en p2f_b0_o UNUSED p2f_b1_en UNUSED UNUSED lssd_scan_in2 UNUSED lssd_scan_out4 p2d_d0_ic2 lssd_scan_in4
22
pwrup_reset_in_ L VDD soc_inn GND UNUSED VDD oob_ad2 GND p2f_a1_en VDDQ f2p_b0_o GND UNUSED GND f2p_b1_en VDDQ p2f_b1_c GND UNUSED VDD UNUSED GND UNUSED VDD p2d_d0_ic4
23
jtag_tdo p2f_a0_o plltest_out oob_ad6 plllock oob_ad3 lssd_ce1_b p2f_a1_e f2p_b0_e f2p_b0_en f2p_b0_on oob_int_lo UNUSED UNUSED f2p_b1_e f2p_b1_o f2p_b1_on p2f_b1_cn lssd_scan_in0 UNUSED lssd_scan_in1 UNUSED lssd_scan_out3 UNUSED lssd_scan_in3
24
p2f_a0_on GND oob_ad7 VDDQ UNUSED GND f2p_a1_cn VDD f2p_a1_o GND oob_devsel1 VDD UNUSED VDDQ UNUSED GND UNUSED VDD p2f_b1_o GND UNUSED VDDQ UNUSED GND UNUSED
25
UNUSED UNUSED plltest_in UNUSED VDDA f2p_a1_c lssd_ce1_a f2p_a1_on oob_wait_L oob_clk oob_devsel2 oob_int_hi UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED lssd_scan_out0 p2f_b1_on lssd_scan_out1 UNUSED lssd_scan_out2 UNUSED UNUSED
A B C D E F G H J K L M N P R T U V W Y AA AB AC AD AE
216
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 35.
Enhanced Port Processor Alpha Pin List
Signal Name
ce0_io ce0_scan ce0_test ce0_tstm3 d2p_d0_id0 d2p_d0_id1 d2p_d0_id2 d2p_d0_id3 d2p_d0_id4 d2p_d0_id5 d2p_d0_id6 d2p_d0_id7 d2p_d0_od0 d2p_d0_od1 d2p_d0_od2 d2p_d0_od3 d2p_d0_od4 d2p_d0_od5 d2p_d0_od6 d2p_d0_od7 d2p_d1_id0 d2p_d1_id1 d2p_d1_id2 d2p_d1_id3 d2p_d1_id4 d2p_d1_id5 d2p_d1_id6 d2p_d1_id7 d2p_d1_od0 d2p_d1_od1 d2p_d1_od2 d2p_d1_od3 d2p_d1_od4 d2p_d1_od5 d2p_d1_od6 d2p_d1_od7 d2p_d2_id0 d2p_d2_id1 d2p_d2_id2 d2p_d2_id3 d2p_d2_id4 d2p_d2_id5 d2p_d2_id6 d2p_d2_id7 f2p_a0_c f2p_a0_cn f2p_a0_e f2p_a0_en f2p_a0_o f2p_a0_on f2p_a1_c f2p_a1_cn f2p_a1_e f2p_a1_en f2p_a1_o f2p_a1_on f2p_b0_c f2p_b0_cn f2p_b0_e f2p_b0_en f2p_b0_o
Pin
A07 C05 A05 C07 AA07 AD05 U10 V09 AC06 AB07 AD09 AE08 G02 K01 J01 J06 H05 H03 J04 L01 AE02 AD03 AC04 AB05 Y07 W08 AA06 AE04 H07 F03 G04 J02 H01 J07 K09 M09 Y05 AA04 W06 W07 AB03 AC02 AD01 AE01 D19 C20 J17 G17 A18 B17 F25 G24 J18 H19 J24 H25 L19 K21 J23 K23 L22
Signal Name
f2p_b0_on f2p_b1_c f2p_b1_cn f2p_b1_e f2p_b1_en f2p_b1_o f2p_b1_on GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND
Pin
L23 U20 T19 R23 R22 T23 U23 B02 B06 B10 B16 B20 B24 D04 D08 D12 D14 D18 D22 F02 F06 F10 F16 F20 F24 H04 H08 H12 H14 H18 H22 K02 K06 K10 K16 K20 K24 L13 M04 M08 M12 M14 M18 M22 N11 N13 N15 P04 P08 P12 P14 P18 P22 R13 T02 T06 T10 T16 T20 T24 V04
Signal Name
GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND jtag_tck jtag_tdi jtag_tdo jtag_tms jtag_trst_L lssd_ce1_a lssd_ce1_b lssd_ce1_c1 lssd_ce1_c2 lssd_ce1_c3 lssd_scan_in0 lssd_scan_in1 lssd_scan_in10 lssd_scan_in11 lssd_scan_in12 lssd_scan_in13 lssd_scan_in14 lssd_scan_in15 lssd_scan_in16 lssd_scan_in17 lssd_scan_in18 lssd_scan_in19 lssd_scan_in2 lssd_scan_in3 lssd_scan_in4 lssd_scan_in5 lssd_scan_in6 lssd_scan_in7 lssd_scan_in8 lssd_scan_in9 lssd_scan_out0 lssd_scan_out1 lssd_scan_out10 lssd_scan_out11 lssd_scan_out12 lssd_scan_out13 lssd_scan_out14 lssd_scan_out15
Pin
V08 V12 V14 V18 V22 Y02 Y06 Y10 Y16 Y20 Y24 AB04 AB08 AB12 AB14 AB18 AB22 AD02 AD06 AD10 AD16 AD20 AD24 C19 E21 A23 A19 G14 G25 G23 M19 M21 P21 W23 AA23 AC03 AC01 AA01 W01 P07 M05 G03 E03 C03 G12 AA21 AE23 AE21 AE19 W14 AA12 AC07 AC05 W25 AA25 AE03 AA05 AA03 W03 P05 M07
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
217
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Signal Name
lssd_scan_out16 lssd_scan_out17 lssd_scan_out18 lssd_scan_out19 lssd_scan_out2 lssd_scan_out3 lssd_scan_out4 lssd_scan_out5 lssd_scan_out6 lssd_scan_out7 lssd_scan_out8 lssd_scan_out9 No Pin oob_ad0 oob_ad1 oob_ad2 oob_ad3 oob_ad4 oob_ad5 oob_ad6 oob_ad7 oob_clk oob_devsel0 oob_devsel1 oob_devsel2 oob_devsel3 oob_int_hi oob_int_lo oob_valid_L oob_wait_L p2d_d0_ic0 p2d_d0_ic1 p2d_d0_ic2 p2d_d0_ic3 p2d_d0_ic4 p2d_d0_ic5 p2d_d0_ic6 p2d_d0_ic7 p2d_d0_id0 p2d_d0_id1 p2d_d0_id2 p2d_d0_id3 p2d_d0_id4 p2d_d0_id5 p2d_d0_id6 p2d_d0_id7 p2d_d0_oc0 p2d_d0_oc1 p2d_d0_oc2 p2d_d0_oc3 p2d_d0_oc4 p2d_d0_oc5 p2d_d0_oc6 p2d_d0_oc7 p2d_d1_ic0 p2d_d1_ic1 p2d_d1_ic2 p2d_d1_ic3 p2d_d1_ic4 p2d_d1_ic5 p2d_d1_ic6
Pin
G01 E01 C01 E12 AC25 AC23 AC21 AC19 AA14 W12 AE07 AE05 A01 G21 K17 G22 F23 G20 G19 D23 C24 K25 M15 L24 L25 L17 M25 M23 L16 J25 V17 U16 AD21 AA19 AE22 AA20 W18 Y19 AC08 AB09 AE11 AD11 W10 U11 R12 AC09 V01 U02 W04 Y03 V07 U08 AA02 AB01 AE20 R17 W17 AA18 AE18 AD17 AB19
Signal Name
p2d_d1_ic7 p2d_d1_id0 p2d_d1_id1 p2d_d1_id2 p2d_d1_id3 p2d_d1_id4 p2d_d1_id5 p2d_d1_id6 p2d_d1_id7 p2d_d1_oc0 p2d_d1_oc1 p2d_d1_oc2 p2d_d1_oc3 p2d_d1_oc4 p2d_d1_oc5 p2d_d1_oc6 p2d_d1_oc7 p2d_d2_ic0 p2d_d2_ic1 p2d_d2_ic2 p2d_d2_ic3 p2d_d2_ic4 p2d_d2_ic5 p2d_d2_ic6 p2d_d2_ic7 p2d_d2_oc0 p2d_d2_oc1 p2d_d2_oc2 p2d_d2_oc3 p2d_d2_oc4 p2d_d2_oc5 p2d_d2_oc6 p2d_d2_oc7 p2d_d3_ic0 p2d_d3_ic1 p2d_d3_ic2 p2d_d3_ic3 p2d_d3_ic4 p2d_d3_ic5 p2d_d3_ic6 p2d_d3_ic7 p2d_d3_oc0 p2d_d3_oc1 p2d_d3_oc2 p2d_d3_oc3 p2d_d3_oc4 p2d_d3_oc5 p2d_d3_oc6 p2d_d3_oc7 p2d_d4_ic0 p2d_d4_ic1 p2d_d4_ic2 p2d_d4_ic3 p2d_d4_ic4 p2d_d4_ic5 p2d_d4_ic6 p2d_d4_ic7 p2d_d4_oc0 p2d_d4_oc1 p2d_d4_oc2 p2d_d4_oc3
Pin
AC20 W09 U09 AE06 AD07 AE10 AE09 AA09 Y09 U07 V05 U01 T01 W02 Y01 R10 W05 AD15 AE15 AB17 AC18 Y17 AA17 AE17 AE16 U03 P11 U06 T07 R02 R01 U04 V03 V15 AA16 AC14 AE14 AC16 AC17 R14 U15 N01 R03 R04 P09 U05 R08 P03 P01 AA13 T13 W15 U14 AC13 AD13 AC15 AB15 L05 L06 R06 R05
Signal Name
p2d_d4_oc4 p2d_d4_oc5 p2d_d4_oc6 p2d_d4_oc7 p2d_d5_ic0 p2d_d5_ic1 p2d_d5_ic2 p2d_d5_ic3 p2d_d5_ic4 p2d_d5_ic5 p2d_d5_ic6 p2d_d5_ic7 p2d_d5_oc0 p2d_d5_oc1 p2d_d5_oc2 p2d_d5_oc3 p2d_d5_oc4 p2d_d5_oc5 p2d_d5_oc6 p2d_d5_oc7 p2d_d6_ic0 p2d_d6_ic1 p2d_d6_ic2 p2d_d6_ic3 p2d_d6_ic4 p2d_d6_ic5 p2d_d6_ic6 p2d_d6_ic7 p2d_d6_oc0 p2d_d6_oc1 p2d_d6_oc2 p2d_d6_oc3 p2d_d6_oc4 p2d_d6_oc5 p2d_d6_oc6 p2d_d6_oc7 p2f_a0_c p2f_a0_cn p2f_a0_e p2f_a0_en p2f_a0_o p2f_a0_on p2f_a1_c p2f_a1_cn p2f_a1_e p2f_a1_en p2f_a1_o p2f_a1_on p2f_b0_c p2f_b0_cn p2f_b0_e p2f_b0_en p2f_b0_o p2f_b0_on p2f_b1_c p2f_b1_cn p2f_b1_e p2f_b1_en p2f_b1_o p2f_b1_on p2s_s0_c
Pin
N07 N06 T05 R07 U12 W11 Y13 W13 AA11 Y11 Y15 AA15 N10 L04 L03 N02 N03 L07 K05 N05 AE12 AC12 AA10 V11 U13 AB11 AC11 AE13 K07 L09 N09 J03 K03 M01 M03 L08 H17 J16 G18 F19 B23 A24 H21 J19 H23 J22 K19 J20 L21 L20 R20 R21 T21 R19 U22 V23 U19 V21 W24 Y25 B13
218
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Signal Name
p2s_s0_cn p2s_s0_e p2s_s0_en p2s_s0_o p2s_s0_on p2s_s1_c p2s_s1_cn p2s_s1_e p2s_s1_en p2s_s1_o p2s_s1_on plllock plltest_in plltest_out pwrup_reset_in_L ref_clk ref_clkn s2p_s0_c s2p_s0_cn s2p_s0_e s2p_s0_en s2p_s0_o s2p_s0_on s2p_s1_c s2p_s1_cn s2p_s1_e s2p_s1_en s2p_s1_o s2p_s1_on soc_in soc_inn soc_out test_di1 test_di2 test_lt test_re test_ri UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED
Pin
C13 E11 F11 F15 E15 J15 G16 D17 C18 A17 A16 E23 C25 C23 A22 C21 A21 C09 C10 E09 F09 C08 D09 C16 C17 D13 A13 J13 H15 D21 C22 E20 E05 A03 E13 H13 E14 A02 A04 A06 A08 A09 A10 A11 A12 A14 A15 A20 A25 B01 B03 B05 B07 B09 B11 B15 B19 B21 B25 C02 C04
Signal Name
UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED
Pin
C06 C11 C12 C14 C15 D01 D03 D05 D07 D11 D15 D25 E02 E04 E06 E07 E08 E10 E16 E17 E18 E19 E22 E24 F01 F05 F07 F13 F17 F21 G05 G06 G07 G08 G09 G10 G11 G13 G15 H09 H11 J05 J08 J11 J12 J14 J21 K11 K13 K15 L02 L12 L14 L18 M17 N04 N16 N17 N18 N19 N20
Signal Name
UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD
Pin
N21 N22 N23 N24 N25 P15 P17 P19 P23 P25 R18 R24 R25 T03 T09 T15 T25 U21 U24 U25 V19 V25 W16 W19 W20 W21 W22 Y21 Y23 AA08 AA22 AA24 AB13 AB21 AB23 AB25 AC10 AC22 AC24 AD19 AD23 AD25 AE24 AE25 B04 B14 B22 D10 D16 F04 F08 F18 F22 H02 H24 K12 K14 L11 L15 M02 M06
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
219
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Signal Name
VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDDA VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ VDDQ
Pin
M10 M13 M16 M20 M24 N12 N14 P06 P10 P13 P16 P20 R11 R15 T12 T14 V02 V24 Y04 Y08 Y18 Y22 AB10 AB16 AD04 AD14 AD22 E25 D24 H20 K18 K22 P24 T18 T22 V20 AB24 V10 V16 Y12 Y14 AB06 AB20 AD08 AD12 AD18 D02 H06 K04 K08 P02 T04 T08 V06 AB02 B08 B12 B18 D06 D20 F12
Signal Name
VDDQ VDDQ VDDQ Vref Vref Vref Vref Vref Vref Vref Vref Vref Vref Vref Vref
Pin
F14 H10 H16 R16 V13 N08 L10 T17 U18 U17 T11 R09 M11 J10 J09
220
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.6.2
Package Dimensions
This section outlines the mechanical dimensions for the Enhanced Port Processor (EPP) device. The package is a 624 ceramic ball grid array (CBGA). NOTE: Drawings are not to scale.
Figure 60. Enhanced Port Processor CBGA Package Dimensions - Top and Side Views
D D1
E1 E
Top View Pin A01 corner A A1
A2
aaa
Side View
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
221
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 61. Enhanced Port Processor (EPP) CBGA Package Dimensions - Bottom View
AE AD AC AB AA Y W V U T R P N M L K J H G F E D C B A 02 04 06 08 10 12 14 16 18 20 22 24 01 03 05 07 09 11 13 15 17 19 21 23 25 Pin A01 corner Bottom View
b
e
Table 36.
Enhanced Port Processor CBGA Mechanical Specifications
e =1.27 mm Symbol Minimum A A1 A2 aaa D D1 E E1 M N b 0.82 NOTE: Ball alloy is 90/10 PbSn Nominal 5.52 4.62 0.90 0.15 32.50 30.48 32.50 30.48 25 x 25 624 0.92 mm Maximum mm mm mm mm mm mm mm mm Units
222
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
3.6.3
Part Number
The PMC-Sierra Part Number for the Enhanced Port Processor is: Enhanced Port Processor PM9315-HC
PMC Logo
0 6K1 5 81 PQ PM9315-HC E n h an c e d P o r t P r o ce ss o r R ev B BXXLLLLLLLL
Country of Origin
Source "B" Part Number PMC Part Number Device Name Revision Lot Number Date Code
BYYW W X XX X XX X XX X XX
Terminal A01 Identifier
PM 9 3 15 - H C - *
PMC prefix IRD prefix IC Chipset family code, 1=TT1, 5 =TTX Sequential part number (1=SCH, 2=XBAR, 3=DS, 5=EPP) Temp. range (C = Commercial 0-70 degrees C) Package code (U = CCGA, H = CBGA) P =prototype, Blank =production part
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
223
Released Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
224
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
4
Crossbar
This chapter contains information on the Crossbar device, part number PM9312-UC, available from PMC-Sierra, Inc. The Crossbar device is a multicast-capable, 32-port, reverse-routed Crossbar. Each Crossbar is connected to the matching logical Dataslice in every port. For example, XBAR0 connects to DS0. Each Crossbar switches an eight byte cell each frame time. The device sets its crosspoints according to the reverse routing tags supplied by the Scheduler, via the EPPs and Dataslices. If the routing tag supplied is non-valid, the frame containing cell data being sent to this DS had a CRC error, or the frame containing the routing tag from this DS had a CRC error, then the VLD bit is not set on the frame to the DS.
4.1
CROSSBAR BLOCKS
Figure 62 shows the basic data flow through the Crossbar. The routing tag sent in the AIB frame from the Dataslice is registered in the routing tag register (RTR). The RTR selects one of the 32 incoming byte-wide busses and the output of the 32:1 byte-wide mux is driven to the transmitter side of the asymmetric serial link dumb end. The total delay from d2x_p[31:0] to x2d_p[31:0] is four cell times.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
225
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 62. The Basic Dataflow Through the Crossbar
RTR
D2x_p{0] x2d_p[0]
d2x_p[1]
RTR
x2d_p[1]
RTR
x2d_p[31] d2x_p[31]
core clocks ref_clk(n) plllock
PLL
OOB Interface Control/Status Registers
JTAG
oob_clk oob_ad[7:0] oob_valid_L oob_wait_L oob_int_hi oob_int_lo oob_devsel[3:0] soc_in(n) soc_out
226
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
jtag_tdi jtag_tms jtag_tck jtag_trst_L jtag_tdo
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
4.1.1
OOB Interface and Control/Status Registers
All of the devices have an OOB interface. This interface allows a single local CPU to control and monitor all of the devices within a core fabric. Internally, each device provides registers that can be mapped into the CPU's address space. These registers are described in more detail in Section 4.4 "Crossbar Registers".
4.2
MODES OF OPERATION
The default mode of operation is for the Crossbar to act as a 32-port switch. In this mode, each ETT1 port has one connection to each of the Crossbars. If an 8- or 16-port switch is required, the Crossbar can be configured so that fewer Crossbar devices are required. In these modes, the term superport represents a system port. In 8-port mode, the Crossbar considers four of its ports to be a single superport. For example, ports 0, 1, 2, and 3 make up superport 0, ports 28, 29, 30, and 31 are superport 7. A single superport is connected to only three Crossbars. For example, superport 0 has its first 4 Dataslices connected to ports 0, 1, 2, and 3 of Crossbar 0, its next 4 Dataslices connected to ports 0, 1, 2, and 3 of Crossbar 1, and its last 4 Dataslices connected to ports 0, 1, 2, and 3 of Crossbar 2. In 16-port mode, the Crossbar considers two of its ports to be a single superport. Ports 0 & 1 make up superport 0, and ports 30 & 31 make up superport 15 for example. A single superport is connected to six Crossbars. In this case, superport 0 has its first 2 Dataslices connected to ports 0 and 1 of Crossbar 0, and its last Dataslices are connected to ports 0 and 1 of Crossbar 5. The mode is set via the OOB bus. The Crossbar acts as if the superports make up a `trunk', so that either two or four of the inputs are routed as a single port.
4.3
OOB ACCESS
The Crossbars can be written and read via the OOB bus. In addition to the device select addresses 0x7FF (all devices) and 0x7B (all Crossbars), each Crossbar can be addressed with an individual device select specified by {000100, CCCC, 000, C} where CCCCC is the Crossbar number in the range 0-31 (these correspond to device selects 0x200 to 0x2F1, not including all addresses within that range). The Registers section of this chapter illustrates the usage of the subaddress field to specify registers within the Crossbar.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
227
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
4.4
CROSSBAR REGISTERS
The Crossbar device select low-order bits are hard wired on the board by four pins (oobdev_sel[3:0]) on the Crossbar package. See Section 1.7.1.3 "Individual Device Selects" on page 79 for more information.
4.4.1
Summary
The following table is a summary of information for all registers in the Crossbar. See the following Descriptions section for more information on individual registers. Read and Clear means that reading the register causes it to be cleared (reset to zero)
Table 37. Crossbar Register Summary
Address 00000h 00004h 00008h 0000Ch 00010h 00014h 00018h 0001Ch 00020h 00024h 00028h 00030h 00034h 0003Ch 00100h
Symbol XSTS XMODERS XIRLMSK XIRHMSK XIR XCRCMSK XCRC XRDYDMSK XRDYDN XRDYUMSK XRDYUP XAIBRS XAIBRDY XAIBEN XPLL Status Control/Reset
Register
Access Read Only Read/Write Read/Write Read/Write Read only Read/Write Read and Clear Read/Write Read and Clear Read/Write Read and Clear Read/Write Read Only Read/Write Read/Write
Default Value 40000000h 00000004h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h FFFFFFFFh 00000000h 00000000h 0001447Ch
Low Priority Mask High Priority Mask General Interrupt Register CRC Error Interrupts Mask CRC Error Interrupts RDY Inactive Interrupts Mask Link RDY Inactive Interrupts Link RDY Active Interrupts Mask Link RDY Active Interrupts AIB Reset AIB Ready AIB Tx Enable PLL Control/Status
228
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
4.4.2
Crossbar Register Descriptions
Read and Clear means that reading the register causes it to be cleared (reset to zero). All bits labeled as Reserved should be set to 0.
4.4.2.1 Symbol:
Status XSTS
Address Offset: 00000h Default Value: 40000000h Access: Read Only The top 8 bits are device_id [3:0] and revision [3:0]. OOB interrupt bits are [1:0] (high, low), all other bits are reserved. Bits 31:28 27:24 23:2 1 0 Device Revision Number. Reserved. High Priority Interrupt. 1 = There is an outstanding high priority interrupt. One of the bits in the General Interrupt Register is set, and is enabled via its corresponding high priority mask. Low Priority Interrupt. 1 = There is an outstanding low priority interrupt. One of the bits in the General Interrupt Register is set, and is enabled via its corresponding low priority mask. Description Device ID Number. Identifies the specific device.
4.4.2.2 Symbol:
Control/Reset XMODERS
Address Offset: 00004h Default Value: 00000004h Access: Read/Write This register configures the number of ports supported by this Crossbar. It also resets the entire device, equivalent to a hardware reset. Bits 31-3 Reserved. Description
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
229
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Bits
Description (Continued) Port Mode. These 2 bits select the number of ports the Crossbar is supporting. A zero value (00) corresponds to 8 ports, a one value (01) corresponds to 16 ports, and a two value (10) corresponds to 32 ports (the default). The number of slices per port are then given by 32/num_ ports_selected. 10 = 32 ports x 1 slice 01 = 16 ports x 2 slices 00 = 8 ports x 4 slices Reset. Writing a 1 to this location will reset the entire device. It is equivalent to a hardware reset. This register is cleared automatically after the device has been reset. Writing a 0 is not necessary. Soft reset takes 1mS to complete.
2:1
0
4.4.2.3 Symbol:
Low Priority Mask XIRLMSK
Address Offset: 00008h Default Value: 00000000h Access: Read/Write Interrupt Mask for interrupts. Bits 31:3 2:0 Reserved. Low Priority Mask. Mask bits for low priority interrupts. Each mask bit is set to 1 to enable a low priority interrupt when the corresponding bit in the General Interrupt Register is 1. Description
4.4.2.4 Symbol:
High Priority Mask XIRHMSK
Address Offset: 0000Ch Default Value: 00000000h Access: Read/Write Interrupt Mask for interrupts. Bits 31:3 2:0 Reserved. High Priority Mask. Mask bits for high priority interrupts. Each mask bit is set to 1 to enable a high priority interrupt when the corresponding bit in the General Interrupt Register is 1. Description
230
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
4.4.2.5 Symbol:
General Interrupt Register XIR
Address Offset: 00010h Default Value: 00000000h Access: Read only Interrupt Register. Bits 31:3 2 Reserved. Ready Active. This is the logical OR of the Ready Active Interrupt register, after it has been masked. This is set if a serial link goes from inactive to active and the corresponding mask bit is 1. This bit is not cleared when read. You must clear the Ready Active Interrupt register. Ready Inactive. This is the logical OR of the Ready Inactive Interrupt register, after it has been masked. This is set if a serial link goes from active to inactive and the corresponding mask bit is 1. This bit is not cleared when read. You must clear the Ready Inactive Interrupt register. CRC Error. This is the logical OR of the CRC register, after it has been masked. This is set if a CRC error occurs and the corresponding mask bit is 1. This bit is not cleared when read. You must clear the CRC Error Interrupts register. Description
1
0
4.4.2.6 Symbol:
CRC Error Interrupts Mask XCRCMSK
Address Offset: 00014h Default Value: 00000000h Access: Read/Write
Interrupt Mask for per-port CRC errors
Bits 31:0
Description CRC Error Interrupts Mask. Each bit is used to mask (enable) interrupts when the corresponding bit in the CRC Error Interrupts register is set to 1. The interrupt is enabled when the bit is 1.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
231
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
4.4.2.7 Symbol:
CRC Error Interrupts XCRC
Address Offset: 00018h Default Value: 00000000h Access: Read and Clear
Interrupt Register for per-port CRC errors
Bits 31:0
Description CRC Error Interrupts. Each bit indicates if a CRC error has occurred on the appropriate link (DS to XBAR) since the last time this register was read. Mask off unwanted (unused ports) bits via the CRC Error Interrupt Mask.
4.4.2.8 Symbol:
RDY Inactive Interrupts Mask XRDYDMSK
Address Offset: 0001Ch Default Value: 00000000h Access: Read/Write
Interrupt Mask for per-port RDY down signals.
Bits 31:0
Description RDY Inactive Interrupts Mask. Each bit is used to mask (enable) interrupts when the corresponding bit in the Link Ready Inactive register is set to 1. The interrupt is enabled when the bit is 1.
232
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
4.4.2.9 Symbol:
Link RDY Inactive Interrupts XRDYDN
Address Offset: 00020h Default Value: 00000000h Access: Read and Clear
Interrupt Register for per-port RDY down signals.
NOTE: The link may be up again (active) by the time the CPU reads this register. Mask off unwanted (unused ports) bits via the Link Ready Inactive Interrupts Mask. Reading this register clears all bits. Bits 31:0 Description Link RDY Inactive Interrupts. Each bit indicates if the Ready signal from a link has transitioned from Active to Inactive (the link has gone down for some reason).
4.4.2.10 Symbol:
Link RDY Active Interrupts Mask XRDYUMSK
Address Offset: 00024h Default Value: 00000000h Access: Read/Write
Interrupt Mask for per-port RDY up signals.
Bits 31:0
Description Link RDY Active Interrupts Mask. Each bit is used to mask (enable) interrupts when the corresponding bit in the Link Ready Active register is set to 1. The interrupt is enabled when the bit is 1.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
233
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
4.4.2.11 Symbol:
Link RDY Active Interrupts XRDYUP
Address Offset: 00028h Default Value: 00000000h Access: Read and Clear
Interrupt Register for per-port RDY up signals.
NOTE: The link may be down again (inactive) by the time the CPU reads this register. Mask off unwanted (unused ports) bits via the Link Ready Active Interrupt Mask. This register is cleared to 0 when read. Bits 31:0 Description Link RDY Active Interrupts. Each bit indicates if the Ready signal from a link has transitioned from Inactive to Active (the link has come up).
4.4.2.12 Symbol:
AIB Reset XAIBRS
Address Offset: 00030h Default Value: FFFFFFFFh Access: Read/Write Resets AIB link for each port. Bits 31:0 Description AIB Reset. Each bit is used to assert reset to the AIB link for each port. Reset is asserted when the bit is 1. If reset, the link will not operate or transition to ready. This register is set to FFFFFFFFh on power-up reset or if the reset bit in the control register is asserted.
234
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
4.4.2.13 Symbol:
AIB Ready XAIBRDY
Address Offset: 00034h Default Value: 00000000h Access: Read Only Indicates the status of the AIB link. Bits 31:0 Description AIB Ready. Each bit indicates if the corresponding AIB link is ready (1 = active) or not (0 = inactive).
4.4.2.14 Symbol:
AIB Tx Enable XAIBEN
Address Offset: 0003Ch Default Value: 00000000h Access: Read/Write Enables the transmitter of the corresponding AIB link. Bits 31:0 Description AIB Tx Enable. If the corresponding port is not physically present in the system then this bit should be set to zero to reduce unwanted electrical noise and to minimize unwanted effects when the port board is inserted.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
235
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
4.4.2.15 Symbol:
PLL Control/Status XPLL
Address Offset: 00100h Default Value: 0001447Ch Access: Read/Write Controls operation of the internal PLL (Phase locked loop). After a power on reset the PLL itself is held in reset. This is reflected in bit 16 of this register. The local CPU must reset this bit to 0 to enable operation of the device, and thus should write 0000447Ch to this register. Bit 31:17 16 15:0 Description PLL status. These bits reflect internal PLL operation status and should be ignored. Reset PLL. When set to 1 the PLL is held reset. The supplied reference clock will be used as the internal clock. The serial links will not be operational. This bit will be 1 after power-up reset and should be deasserted for normal operation. PLL reset takes 10mS to complete. PLL control.
236
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
4.5
CROSSBAR SIGNAL DESCRIPTIONS
This section describes the Crossbar signals. The following notation is used to describe the signal type: I O B Input pin Output pin Bidirectional Input/Output pin
The signal description also includes the type of buffer used for the particular signal: PECL STI PECL are Pseudo-ECL (positive voltage ECL) compatible signals. STI is a very high-speed asynchronous communications protocol used to connect devices that may be 1 to 15 meters apart. All HSTL specifications are consistent with EIA/JEDEC Standard, EIA/JESD8-6 "High Speed Transceivers Logic (HSTL): A 1.5V Output Buffer Supply Voltage based Interface Standard for Digital Integrated Circuits", dated 8/95. Refer to these specifications for more information. The CMOS Buffers are either 3.3 V compatible or 2.5 V Low Voltage TTL compatible signals, as noted.
Table 38. Crossbar Signal Descriptions
HSTL
CMOS
Name
I/O
Type OOB Interface
Description
oob_ad[7:0] oob_clk oob_devsel0 oob_devsel1 oob_devsel2 oob_devsel3 oob_int_hi oob_int_lo oob_valid_L
B I I I I I O O I
CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS
OOB Address bus OOB Clock OOB device select OOB device select OOB device select OOB device select OOB Interrupt OOB Interrupt OOB Control
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
237
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 38. Crossbar Signal Descriptions (Continued)
Name oob_wait_L
I/O O
Type CMOS OOB Control AIB Interface
Description
d2x_p[31:0]_[c, cn, e, en, o, on] x2d_p[31:0]_[c, cn, e, en, o, on]
I O
STI STI
DS to Xbar cell data Xbar to DS cell data
Power Supply, Clock Source, Reset, and Diagnostics plllock pwruprst_L soc_out VDDA VDD GND ref_clk and ref_ clkn soc_in and soc_ inn I I O I O CMOS CMOS CMOS Isolated Supply Supply Supply PECL PECL Test output Synchronous, Active Low Reset Buffered soc_in (NC) VDD (2.5 V) for PLL VDD (2.5 V) GND (0 V) System 200MHz clock (Differential) System Start-of-cell (Differential) JTAG Interface jtag_tck jtag_tdi jtag_tdo jtag_tms jtag_trst_L I I O I I CMOS_ 2.5_only CMOS_ 2.5_only CMOS_ 2.5_only CMOS_ 2.5_only CMOS_ 2.5_only 1149.1 JTAG 2.5V ONLY! 1149.1 JTAG 2.5V ONLY! 1149.1 JTAG 2.5V ONLY! 1149.1 JTAG 2.5V ONLY! 1149.1 JTAG 2.5V ONLY!
238
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 38. Crossbar Signal Descriptions (Continued)
Name
I/O
Type
Description
ASIC Manufacturing Test Interface ce0_io ce0_scan lssd_ce1_a lssd_ce1_b lssd_ce1_c1 lssd_ce1_c2 lssd_scan_ in[15:0] lssd_scan_ out[15:0] M_ackreg_L M_pending_L M_xreset_L plltest_in plltest_out test_di1 test_di2 test_lt test_ri I I I I I I I O O O O I O I I I I CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS Test (GND) Test (GND) Test (VDD) Test (VDD) Test (VDD) Test (VDD) Test: Scan input (GND) Test: Scan output Test (NC) Test (NC) Test (NC) Test (GND) Test output (NC) Test (VDD) Should be driven to GND during reset. All outputs are tristated when low. Test (VDD) Should be driven to GND during reset. All outputs are tristated when low. Test (VDD) Test (VDD)
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
239
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
4.6 4.6.1
PINOUT AND PACKAGE INFORMATION Pinout Tables
Table 39. Crossbar Pinout (left side)
01 A B C D E F G H J K L M N P R T U V W Y AA AB AC AD AE AF AG AH AJ AK AL AM AN
No Pin UNUSED ref_clkn x2d_p31_e VDDA d2x_p31_o
02
UNUSED GND UNUSED VDD x2d_p31_en GND
03
test_di2 UNUSED plllock UNUSED plltest_in UNUSED
04
x2d_p0_en VDD UNUSED GND UNUSED VDD UNUSED GND
05
x2d_p0_e lssd_scan_ out6 UNUSED ref_clk UNUSED UNUSED UNUSED
06
GND soc_inn VDD UNUSED GND UNUSED VDD
07
d2x_p0_o
08
VDD
09
UNUSED x2d_p1_e x2d_p1_o d2x_p0_c
10
d2x_p1_on GND x2d_p1_on VDD d2x_p0_cn GND x2d_p1_cn VDD oob_clk GND
11
UNUSED d2x_p1_o x2d_p2_o d2x_p1_cn d2x_p1_c x2d_p2_cn d2x_p1_e x2d_p1_c d2x_p0_en x2d_p0_cn UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED
lssd_scan_ lssd_scan_ d2x_p0_on x2d_p1_en out5 out7 lssd_scan_ x2d_p0_on out8 pwruprst_L soc_in UNUSED UNUSED UNUSED GND
oob_valid_L x2d_p0_o VDD soc_out GND d2x_p0_e x2d_p0_c oob_ devsel3 UNUSED UNUSED
UNUSED d2x_p31_on plltest_out x2d_p30_e VDD x2d_p31_o
UNUSED x2d_p30_en x2d_p30_on d2x_p31_cn x2d_p31_on d2x_p31_en x2d_p31_cn UNUSED d2x_p30_o GND x2d_p30_o VDD d2x_p31_c GND x2d_p30_c VDD
UNUSED d2x_p30_on x2d_p29_on d2x_p30_c d2x_p30_cn x2d_p29_c d2x_p30_en x2d_p30_cn d2x_p31_e x2d_p31_c x2d_p29_e VDD d2x_p29_cn GND x2d_p29_o VDD d2x_p29_e GND d2x_p30_e VDD
UNUSED x2d_p29_en x2d_p28_o x2d_p28_on d2x_p29_c d2x_p28_c x2d_p28_e d2x_p29_en x2d_p29_cn UNUSED d2x_p29_o GND x2d_p28_c VDD x2d_p28_cn GND UNUSED VDD UNUSED VDD UNUSED GND d2x_p28_cn VDD x2d_p28_en GND UNUSED VDD UNUSED VDD
UNUSED d2x_p29_on d2x_p28_o d2x_p28_on UNUSED UNUSED UNUSED UNUSED UNUSED x2d_p26_en UNUSED d2x_p26_on UNUSED x2d_p25_en UNUSED d2x_p25_on UNUSED x2d_p24_en UNUSED d2x_p24_on UNUSED UNUSED UNUSED VDD UNUSED VDD UNUSED UNUSED UNUSED GND UNUSED GND UNUSED UNUSED UNUSED UNUSED d2x_p27_c
UNUSED d2x_p28_en d2x_p28_e lssd_scan_ out2 UNUSED lssd_scan_ out1 UNUSED x2d_p27_c GND UNUSED GND lssd_scan_ out3 UNUSED lssd_scan_ out0
x2d_p26_e x2d_p27_en x2d_p27_e GND d2x_p27_cn VDD
x2d_p27_o x2d_p27_on UNUSED VDD d2x_p27_o GND UNUSED VDD
d2x_p26_o d2x_p27_en d2x_p27_e x2d_p26_cn x2d_p27_cn d2x_p27_on x2d_p26_o d2x_p26_c VDD x2d_p26_c GND d2x_p26_en VDD x2d_p26_on GND x2d_p25_on
x2d_p25_e d2x_p26_e x2d_p25_cn x2d_p25_c d2x_p26_cn x2d_p25_o d2x_p25_c x2d_p24_on d2x_p24_cn UNUSED GND d2x_p25_en VDD x2d_p24_cn GND d2x_p25_cn VDD UNUSED GND UNUSED UNUSED UNUSED GND UNUSED VDD x2d_p23_c d2x_p23_e x2d_p22_cn
d2x_p25_o d2x_p25_e x2d_p24_c d2x_p24_e x2d_p24_o d2x_p24_c VDD x2d_p24_e GND d2x_p24_o VDD UNUSED GND UNUSED d2x_p24_en UNUSED UNUSED UNUSED UNUSED lssd_scan_ in14 UNUSED GND UNUSED VDD UNUSED GND UNUSED VDD UNUSED UNUSED UNUSED UNUSED UNUSED lssd_scan_ in12 x2d_p23_en VDD UNUSED GND UNUSED VDD UNUSED GND UNUSED UNUSED UNUSED UNUSED UNUSED
UNUSED x2d_p23_cn x2d_p22_c d2x_p22_en VDD d2x_p23_en GND x2d_p21_c
UNUSED x2d_p23_on d2x_p23_c d2x_p22_cn GND d2x_p23_cn VDD d2x_p22_c
lssd_scan_ x2d_p23_o x2d_p22_on x2d_p22_o x2d_p21_on in10 d2x_p23_on VDD x2d_p22_en UNUSED GND d2x_p22_o d2x_p22_on UNUSED
lssd_scan_ lssd_scan_ lssd_scan_ x2d_p22_e d2x_p23_o x2d_p23_e in11 in13 in15
240
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
PMC-2000164
ISSUE 3
ENHANCED TT1TM SWITCH FABRIC
Table 40.
Crossbar Pinout (center)
12 A B C D E F G H J K L M N P R T U V W Y AA AB AC AD AE AF AG AH AJ AK AL AM AN
x2d_p2_en VDD d2x_p2_c GND
13
UNUSED x2d_p2_e
14
GND
15
d2x_p2_o
16
UNUSED VDD UNUSED GND UNUSED VDD lssd_scan_ out10 GND
17
UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED GND UNUSED VDD GND VDD UNUSED GND UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED
18
UNUSED VDD UNUSED GND UNUSED VDD lssd_scan_ out11 GND lssd_scan_ out12 VDD oob_ad1 GND UNUSED VDD UNUSED GND VDD GND UNUSED VDD UNUSED GND UNUSED VDD
19
UNUSED x2d_p5_en x2d_p4_e x2d_p4_en UNUSED oob_ad2 oob_ad3 x2d_p4_on x2d_p4_o oob_ad0 UNUSED UNUSED UNUSED UNUSED GND UNUSED UNUSED UNUSED GND UNUSED UNUSED UNUSED UNUSED UNUSED
20
x2d_p5_e GND d2x_p4_c VDD d2x_p4_cn GND x2d_p4_cn VDD d2x_p4_on GND UNUSED VDD UNUSED GND UNUSED VDD GND VDD UNUSED GND UNUSED VDD UNUSED GND
21
UNUSED d2x_p5_on d2x_p4_e d2x_p4_en x2d_p5_c x2d_p4_c d2x_p4_o x2d_p5_on d2x_p5_cn UNUSED UNUSED UNUSED VDD UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED VDD UNUSED UNUSED UNUSED
22
d2x_p5_o VDD x2d_p5_cn GND d2x_p5_e VDD x2d_p5_o GND x2d_p6_o VDD M_xreset_L GND UNUSED VDD UNUSED GND UNUSED GND UNUSED VDD UNUSED GND UNUSED VDD
d2x_p2_on UNUSED
x2d_p3_on x2d_p3_cn d2x_p3_on x2d_p3_o VDD x2d_p3_c GND d2x_p3_c VDD d2x_p3_o UNUSED oob_ad5 oob_ad4 d2x_p3_e
x2d_p2_on d2x_p2_cn VDD d2x_p3_cn
d2x_p2_en x2d_p3_en GND d2x_p1_en VDD oob_ devsel2 GND UNUSED VDD UNUSED GND UNUSED GND UNUSED VDD UNUSED GND UNUSED VDD d2x_p2_e x2d_p2_c oob_int_hi UNUSED oob_ devsel1 VDD UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED VDD UNUSED UNUSED UNUSED x2d_p21_ cn d2x_p21_ en
lssd_scan_ x2d_p3_e d2x_p3_en out9 GND oob_int_lo VDD oob_ devsel0 GND UNUSED VDD GND VDD UNUSED GND UNUSED VDD UNUSED GND oob_ad7 UNUSED oob_wait_L UNUSED UNUSED GND UNUSED UNUSED UNUSED GND UNUSED UNUSED UNUSED UNUSED UNUSED VDD oob_ad6 GND UNUSED VDD UNUSED GND VDD GND UNUSED VDD UNUSED GND UNUSED VDD
lssd_scan_ x2d_p20_ d2x_p20_e d2x_p22_e in9 en d2x_p20_ GND VDD GND en d2x_p20_ lssd_scan_ d2x_p21_e x2d_p20_e UNUSED cn in8 VDD d2x_p20_c GND x2d_p20_ cn VDD UNUSED UNUSED d2x_p20_ on VDD UNUSED GND UNUSED VDD UNUSED
lssd_scan_ x2d_p19_on d2x_p19_o in6 GND lssd_scan_ in7 VDD UNUSED GND UNUSED VDD UNUSED x2d_p19_o UNUSED UNUSED UNUSED x2d_p19_e VDD
d2x_p18_c x2d_p17_on x2d_p18_o GND
x2d_p19_c d2x_p19_on x2d_p18_on GND x2d_p19_cn VDD
x2d_p21_o d2x_p21_c GND d2x_p21_ cn VDD x2d_p21_e x2d_p20_ on
d2x_p19_c x2d_p18_cn d2x_p18_en VDD d2x_p19_e GND
x2d_p20_o x2d_p20_c d2x_p20_o x2d_p21_ en UNUSED GND d2x_p21_ on
x2d_p19_en d2x_p19_cn d2x_p19_en x2d_p18_c x2d_p18_e UNUSED GND x2d_p18_en d2x_p18_o UNUSED VDD d2x_p18_on
d2x_p21_o UNUSED
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
241
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 41.
Crossbar Pinout (right side)
23 A B C D E F G H J K L M N P R T U V W Y AA AB AC AD AE AF AG AH AJ AK AL AM AN
UNUSED x2d_p6_en d2x_p5_en x2d_p6_c x2d_p6_cn d2x_p5_c x2d_p6_on d2x_p6_cn
24
x2d_p6_e GND d2x_p6_e VDD x2d_p7_c GND d2x_p6_c VDD
25
UNUSED d2x_p6_on d2x_p6_en x2d_p7_cn d2x_p7_en x2d_p7_on d2x_p7_cn M_pending_ L
26
d2x_p6_o VDD d2x_p7_e GND UNUSED VDD UNUSED GND UNUSED VDD x2d_p9_c GND
27
lssd_scan_ out14 x2d_p7_en lssd_scan_ out13 UNUSED UNUSED UNUSED UNUSED UNUSED x2d_p8_c x2d_p9_cn
28
x2d_p7_e GND UNUSED VDD UNUSED GND UNUSED VDD d2x_p8_e GND
29
lssd_scan_ out4 d2x_p7_on lssd_scan_ out15 UNUSED UNUSED UNUSED UNUSED UNUSED x2d_p8_o d2x_p8_cn
30
d2x_p7_o VDD UNUSED GND UNUSED VDD UNUSED GND d2x_p8_c VDD
31
jtag_tdo UNUSED jtag_tms UNUSED jtag_tck UNUSED jtag_trst_L x2d_p8_on x2d_p9_o x2d_p9_on
32
UNUSED GND UNUSED VDD x2d_p8_e GND d2x_p8_o VDD x2d_p9_e GND d2x_p9_o VDD
33
UNUSED UNUSED jtag_tdi x2d_p8_en UNUSED d2x_p8_on UNUSED x2d_p9_en UNUSED d2x_p9_on UNUSED x2d_p10_en UNUSED d2x_p10_on UNUSED UNUSED UNUSED UNUSED
x2d_p7_o M_ackreg_L UNUSED d2x_p7_c UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED d2x_p16_cn GND UNUSED
x2d_p8_cn d2x_p8_en VDD UNUSED GND UNUSED VDD UNUSED VDD UNUSED GND UNUSED VDD d2x_p9_ en
d2x_p9_e x2d_p10_cn d2x_p9_c d2x_p10_en VDD x2d_p10_on
d2x_p9_cn x2d_p10_o GND d2x_p10_c
x2d_p10_c d2x_p10_e x2d_p11_en d2x_p11_cn d2x_p10_cn x2d_p11_o x2d_p11_on x2d_p10_e x2d_p11_e VDD d2x_p11_c UNUSED ce0_scan test_lt test_di1 GND UNUSED VDD UNUSED VDD UNUSED GND x2d_p11_c UNUSED UNUSED UNUSED UNUSED VDD d2x_p11_o GND UNUSED GND x2d_p11_cn d2x_p11_ on UNUSED UNUSED UNUSED GND d2x_p10_o VDD UNUSED VDD
d2x_p11_en d2x_p11_e ce0_io UNUSED test_ri GND UNUSED GND
x2d_p12_o x2d_p12_on UNUSED d2x_p12_on VDD x2d_p12_cn
UNUSED x2d_p12_en x2d_p12_e x2d_p13_en UNUSED d2x_p12_cn VDD d2x_p12_c GND x2d_p13_e
d2x_p13_ x2d_p13_on d2x_p12_o x2d_p12_c x2d_p13_c d2x_p12_en d2x_p12_e d2x_p13_on UNUSED cn x2d_p13_ x2d_p14_o GND x2d_p13_o VDD d2x_p13_e GND VDD d2x_p13_o cn
d2x_p15_c x2d_p15_o d2x_p14_cn x2d_p14_on d2x_p13_c x2d_p14_cn x2d_p14_c d2x_p13_en x2d_p14_en UNUSED GND UNUSED UNUSED UNUSED VDD UNUSED GND UNUSED VDD UNUSED GND d2x_p14_c GND x2d_p15_c VDD d2x_p14_e GND x2d_p14_e
x2d_p16_on UNUSED d2x_p17_c VDD
d2x_p15_cn x2d_p15_on d2x_p15_en x2d_p15_cn d2x_p14_en d2x_p14_on UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED VDD UNUSED GND UNUSED VDD UNUSED UNUSED UNUSED lssd_ce1_ c1 UNUSED GND UNUSED VDD UNUSED GND d2x_p15_e VDD d2x_p14_o
x2d_p17_o d2x_p17_cn d2x_p16_c d2x_p18_cn GND x2d_p16_o
lssd_ce1_a x2d_p15_en UNUSED UNUSED GND x2d_p15_e
x2d_p17_c x2d_p16_cn d2x_p16_e x2d_p17_ cn VDD x2d_p16_c
lssd_ce1_b d2x_p15_on UNUSED UNUSED VDD UNUSED GND UNUSED d2x_p15_o lssd_ce1_ c2 UNUSED UNUSED
lssd_scan_ lssd_scan_ lssd_scan_ d2x_p18_e d2x_p17_en d2x_p17_e d2x_p16_en UNUSED UNUSED in5 in3 in1 x2d_p17_ GND d2x_p17_o VDD x2d_p16_e GND d2x_p16_o VDD UNUSED e lssd_scan_ lssd_scan_ lssd_scan_ d2x_p16_on x2d_p16_en UNUSED x2d_p17_en UNUSED d2x_p17_on in0 in2 in4
242
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 33.
Crossbar Alpha Pin List
Signal Name
ce0_io ce0_scan d2x_p0_c d2x_p0_cn d2x_p0_e d2x_p0_en d2x_p0_o d2x_p0_on d2x_p10_c d2x_p10_cn d2x_p10_e d2x_p10_en d2x_p10_o d2x_p10_on d2x_p11_c d2x_p11_cn d2x_p11_e d2x_p11_en d2x_p11_o d2x_p11_on d2x_p12_c d2x_p12_cn d2x_p12_e d2x_p12_en d2x_p12_o d2x_p12_on d2x_p13_c d2x_p13_cn d2x_p13_e d2x_p13_en d2x_p13_o d2x_p13_on d2x_p14_c d2x_p14_cn d2x_p14_e d2x_p14_en d2x_p14_o d2x_p14_on d2x_p15_c d2x_p15_cn d2x_p15_e d2x_p15_en d2x_p15_o d2x_p15_on d2x_p16_c d2x_p16_cn d2x_p16_e d2x_p16_en d2x_p16_o d2x_p16_on d2x_p17_c d2x_p17_cn d2x_p17_e d2x_p17_en d2x_p17_o d2x_p17_on d2x_p18_c d2x_p18_cn d2x_p18_e d2x_p18_en d2x_p18_o
Pin
T25 T27 D09 E10 F09 J11 B07 A06 M31 N29 N26 M27 R32 P33 P27 N28 R26 R25 R30 R31 Y31 Y29 AA31 AA30 AA27 Y25 AC28 AA25 AB29 AC31 AB33 AA32 AD27 AC26 AD31 AE31 AF33 AE32 AC24 AE27 AF31 AE29 AK33 AJ32 AG25 AD23 AJ25 AL26 AM29 AN30 AF23 AG24 AL25 AL24 AM25 AN26 AE21 AH23 AL23 AJ22 AM21
Signal Name
d2x_p18_on d2x_p19_c d2x_p19_cn d2x_p19_e d2x_p19_en d2x_p19_o d2x_p19_on d2x_p1_c d2x_p1_cn d2x_p1_e d2x_p1_en d2x_p1_o d2x_p1_on d2x_p20_c d2x_p20_cn d2x_p20_e d2x_p20_en d2x_p20_o d2x_p20_on d2x_p21_c d2x_p21_cn d2x_p21_e d2x_p21_en d2x_p21_o d2x_p21_on d2x_p22_c d2x_p22_cn d2x_p22_e d2x_p22_en d2x_p22_o d2x_p22_on d2x_p23_c d2x_p23_cn d2x_p23_e d2x_p23_en d2x_p23_o d2x_p23_on d2x_p24_c d2x_p24_cn d2x_p24_e d2x_p24_en d2x_p24_o d2x_p24_on d2x_p25_c d2x_p25_cn d2x_p25_e d2x_p25_en d2x_p25_o d2x_p25_on d2x_p26_c d2x_p26_cn d2x_p26_e d2x_p26_en d2x_p26_o d2x_p26_on d2x_p27_c d2x_p27_cn d2x_p27_e d2x_p27_en d2x_p27_o d2x_p27_on
Pin
AN22 AJ20 AL20 AK21 AL21 AE20 AG21 E11 D11 G11 J12 B11 A10 AH13 AG14 AE15 AF15 AL15 AK15 AJ13 AL12 AG12 AF13 AN14 AM15 AK11 AJ11 AE12 AG11 AN10 AM11 AJ10 AK09 AE11 AH09 AN06 AM07 AE07 AC10 AE05 AF03 AJ02 AK01 AC08 AD07 AE03 AD03 AE02 AF01 AA09 AC06 AC03 AB05 AA02 AB01 Y05 Y03 AA04 AA03 Y09 AA07
Signal Name
d2x_p28_c d2x_p28_cn d2x_p28_e d2x_p28_en d2x_p28_o d2x_p28_on d2x_p29_c d2x_p29_cn d2x_p29_e d2x_p29_en d2x_p29_o d2x_p29_on d2x_p2_c d2x_p2_cn d2x_p2_e d2x_p2_en d2x_p2_o d2x_p2_on d2x_p30_c d2x_p30_cn d2x_p30_e d2x_p30_en d2x_p30_o d2x_p30_on d2x_p31_c d2x_p31_cn d2x_p31_e d2x_p31_en d2x_p31_o d2x_p31_on d2x_p3_c d2x_p3_cn d2x_p3_e d2x_p3_en d2x_p3_o d2x_p3_on d2x_p4_c d2x_p4_cn d2x_p4_e d2x_p4_en d2x_p4_o d2x_p4_on d2x_p5_c d2x_p5_cn d2x_p5_e d2x_p5_en d2x_p5_o d2x_p5_on d2x_p6_c d2x_p6_cn d2x_p6_e d2x_p6_en d2x_p6_o d2x_p6_on d2x_p7_c d2x_p7_cn d2x_p7_e d2x_p7_en d2x_p7_o d2x_p7_on d2x_p8_c
Pin
N06 P07 R09 R08 R03 R04 N05 M03 M07 N08 P01 R02 C12 E13 H13 G12 B15 A14 L04 L05 M09 L07 K01 L02 K05 J04 L09 J06 F01 G02 G14 F13 H15 J15 D15 C15 C20 E20 C21 D21 G21 J20 F23 J21 E22 C23 A22 B21 G24 H23 C24 C25 A26 B25 K23 G25 C26 E25 A30 B29 J30
243
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Signal Name
d2x_p8_cn d2x_p8_e d2x_p8_en d2x_p8_o d2x_p8_on d2x_p9_c d2x_p9_cn d2x_p9_e d2x_p9_en d2x_p9_o d2x_p9_on GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND
Pin
K29 J28 L25 G32 F33 L29 L30 L27 M25 L32 K33 B02 B06 B10 B14 B20 B24 B28 B32 D04 D08 D12 D16 D18 D22 D26 D30 F02 F06 F10 F14 F20 F24 F28 F32 H04 H08 H12 H16 H18 H22 H26 H30 K02 K06 K10 K14 K20 K24 K28 K32 M04 M08 M12 M16 M18 M22 M26 M30 P02 P06
Signal Name
GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND
Pin
P10 P14 P17 P20 P24 P28 P32 R15 R19 T04 T08 T12 T16 T18 T22 T26 T30 U14 U17 U20 V04 V08 V12 V16 V18 V22 V26 V30 W15 W19 Y02 Y06 Y10 Y14 Y17 Y20 Y24 Y28 Y32 AB04 AB08 AB12 AB16 AB18 AB22 AB26 AB30 AD02 AD06 AD10 AD14 AD20 AD24 AD28 AD32 AF04 AF08 AF12 AF16 AF18 AF22
Signal Name
GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND jtag_tck jtag_tdi jtag_tdo jtag_tms jtag_trst_L lssd_ce1_a lssd_ce1_b lssd_ce1_c1 lssd_ce1_c2 lssd_scan_in0 lssd_scan_in1 lssd_scan_in10 lssd_scan_in11 lssd_scan_in12 lssd_scan_in13 lssd_scan_in14 lssd_scan_in15 lssd_scan_in2 lssd_scan_in3 lssd_scan_in4 lssd_scan_in5 lssd_scan_in6 lssd_scan_in7 lssd_scan_in8 lssd_scan_in9 lssd_scan_out0 lssd_scan_out1 lssd_scan_out10 lssd_scan_out11 lssd_scan_out12 lssd_scan_out13 lssd_scan_out14 lssd_scan_out15 lssd_scan_out2 lssd_scan_out3
Pin
AF26 AF30 AH02 AH06 AH10 AH14 AH20 AH24 AH28 AH32 AK04 AK08 AK12 AK16 AK18 AK22 AK26 AK30 AM02 AM06 AM10 AM14 AM20 AM24 AM28 AM32 E31 C33 A31 C31 G31 AG31 AJ31 AJ29 AL33 AN31 AL31 AL07 AN07 AL05 AN05 AL03 AN03 AN29 AL29 AN27 AL27 AE18 AG18 AG16 AE16 V09 V07 G16 G18 J18 C27 A27 C29 T07 T09
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
244
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Signal Name
lssd_scan_out4 lssd_scan_out5 lssd_scan_out6 lssd_scan_out7 lssd_scan_out8 lssd_scan_out9 M_ackreg_L M_pending_L M_xreset_L No Pin oob_ad0 oob_ad1 oob_ad2 oob_ad3 oob_ad4 oob_ad5 oob_ad6 oob_ad7 oob_clk oob_devsel0 oob_devsel1 oob_devsel2 oob_devsel3 oob_int_hi oob_int_lo oob_valid_L oob_wait_L plllock plltest_in plltest_out pwruprst_L ref_clk ref_clkn soc_in soc_inn soc_out test_di1 test_di2 test_lt test_ri UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED
Pin
A29 A05 C05 A07 C07 J16 J24 H25 L22 A01 K19 L18 F19 G19 G15 F15 L16 K15 J10 N14 M13 L12 H09 K13 L14 E08 M15 C03 E03 G03 D07 E05 C01 E07 C06 G08 V27 A03 U27 V25 A02 A09 A11 A13 A15 A16 A17 A18 A19 A21 A23 A25 A32 A33 B01 B03 B17 B31 B33 C02 C04
Signal Name
UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED
Pin
C16 C17 C18 C28 C30 C32 D03 D05 D17 D27 D29 D31 E04 E06 E15 E16 E17 E18 E19 E26 E27 E28 E29 E30 E33 F03 F05 F07 F17 F27 F29 F31 G01 G04 G05 G06 G07 G17 G26 G27 G28 G29 G30 G33 H05 H07 H17 H27 H29 J01 J08 J09 J17 J25 J26 J33 K09 K17 K21 K25 L01
Signal Name
UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED
Pin
L11 L13 L15 L17 L19 L20 L21 L23 L33 M11 M17 M19 M21 M23 N01 N10 N11 N12 N15 N16 N17 N18 N19 N20 N22 N23 N24 N33 P11 P13 P15 P19 P21 P23 R01 R05 R06 R07 R10 R11 R12 R13 R14 R16 R17 R18 R20 R21 R22 R23 R24 R27 R28 R29 R33 T01 T03 T05 T11 T13 T15
245
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Signal Name
UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED
Pin
T19 T21 T23 T29 T31 T33 U01 U02 U03 U04 U05 U06 U07 U08 U09 U10 U11 U12 U13 U15 U19 U21 U22 U23 U24 U25 U26 U28 U29 U30 U31 U32 U33 V01 V03 V05 V11 V13 V15 V19 V21 V23 V29 V31 V33 W01 W05 W06 W07 W10 W11 W12 W13 W14 W16 W17 W18 W20 W21 W22 W23
Signal Name
UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED
Pin
W24 W27 W28 W29 W33 Y11 Y13 Y15 Y19 Y21 Y23 AA01 AA10 AA11 AA12 AA14 AA15 AA16 AA17 AA18 AA19 AA20 AA22 AA23 AA24 AA33 AB11 AB13 AB15 AB17 AB19 AB21 AB23 AC01 AC11 AC12 AC13 AC14 AC15 AC16 AC17 AC18 AC19 AC20 AC21 AC22 AC23 AC33 AD09 AD13 AD15 AD17 AD19 AD21 AD25 AE01 AE08 AE09 AE10 AE17 AE24
Signal Name
UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED
Pin
AE25 AE26 AE33 AF05 AF07 AF09 AF17 AF25 AF27 AF29 AG01 AG03 AG04 AG05 AG06 AG07 AG08 AG15 AG17 AG19 AG26 AG27 AG28 AG29 AG30 AG33 AH03 AH05 AH07 AH15 AH17 AH19 AH27 AH29 AH31 AJ01 AJ03 AJ04 AJ05 AJ06 AJ07 AJ08 AJ15 AJ16 AJ17 AJ18 AJ19 AJ26 AJ27 AJ28 AJ30 AJ33 AK03 AK05 AK07 AK17 AK27 AK29 AK31 AL01 AL02
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
246
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Signal Name
UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD
Pin
AL04 AL06 AL16 AL17 AL18 AL28 AL30 AL32 AM01 AM03 AM17 AM31 AM33 AN01 AN02 AN09 AN11 AN13 AN15 AN16 AN17 AN18 AN19 AN21 AN23 AN25 AN32 AN33 B04 B12 B18 B26 D10 D20 D28 D32 F04 F08 F16 F22 H02 H14 H24 H28 K08 K12 K18 K30 M06 M24 M32 N13 N21 P04 P16 P18 P26 T02 T10 T14 T17
Signal Name
VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD
Pin
T20 T28 U16 U18 V06 V14 V17 V20 V24 V32 Y08 Y16 Y18 Y30 AA13 AA21 AB02 AB10 AB28 AD04 AD16 AD22 AD26 AF06 AF10 AF20 AF32 AH12 AH18 AH26 AH30 AK02 AK06 AK14 AK24 AM08 AM16 AM22 AM30 F30 H32 K26 M28 P22 P30 T24 T32 V28 Y22 Y26 AB24 AB32 AD30 AF28 AK32 AB14 AB20 AD12 AD18 AF14 AF24
Signal Name
VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDDA x2d_p0_c x2d_p0_cn x2d_p0_e x2d_p0_en x2d_p0_o x2d_p0_on x2d_p10_c x2d_p10_cn x2d_p10_e x2d_p10_en x2d_p10_o x2d_p10_on x2d_p11_c x2d_p11_cn x2d_p11_e x2d_p11_en x2d_p11_o x2d_p11_on
Pin
AH08 AH16 AH22 AK10 AK20 AK28 AM04 AM12 AM18 AM26 D02 H06 K04 M02 M10 P08 P12 T06 V02 V10 Y04 Y12 AB06 AD08 AF02 AH04 B08 B16 B22 B30 D06 D14 D24 F12 F18 F26 H10 H20 K16 K22 M14 M20 E01 G09 K11 B05 A04 E09 C08 N25 L28 N32 M33 L31 M29 P29 P31 P25 N27 N30 N31
247
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Signal Name
x2d_p12_c x2d_p12_cn x2d_p12_e x2d_p12_en x2d_p12_o x2d_p12_on x2d_p13_c x2d_p13_cn x2d_p13_e x2d_p13_en x2d_p13_o x2d_p13_on x2d_p14_c x2d_p14_cn x2d_p14_e x2d_p14_en x2d_p14_o x2d_p14_on x2d_p15_c x2d_p15_cn x2d_p15_e x2d_p15_en x2d_p15_o x2d_p15_on x2d_p16_c x2d_p16_cn x2d_p16_e x2d_p16_en x2d_p16_o x2d_p16_on x2d_p17_c x2d_p17_cn x2d_p17_e x2d_p17_en x2d_p17_o x2d_p17_on x2d_p18_c x2d_p18_cn x2d_p18_e x2d_p18_en x2d_p18_o x2d_p18_on x2d_p19_c x2d_p19_cn x2d_p19_e x2d_p19_en x2d_p19_o x2d_p19_on x2d_p1_c x2d_p1_cn x2d_p1_e x2d_p1_en x2d_p1_o x2d_p1_on x2d_p20_c x2d_p20_cn x2d_p20_e x2d_p20_en x2d_p20_o x2d_p20_on x2d_p21_c
Pin
AA28 Y27 W31 W30 W25 W26 AA29 AB31 Y33 W32 AB27 AA26 AC30 AC29 AD33 AC32 AB25 AC27 AD29 AE30 AH33 AG32 AC25 AE28 AK25 AJ24 AM27 AN28 AH25 AE23 AJ23 AK23 AM23 AN24 AG23 AE22 AL22 AJ21 AM19 AN20 AF21 AG22 AG20 AH21 AK19 AL19 AF19 AE19 H11 G10 B09 A08 C09 C10 AL14 AJ14 AG13 AE14 AL13 AK13 AH11
Signal Name
x2d_p21_cn x2d_p21_e x2d_p21_en x2d_p21_o x2d_p21_on x2d_p22_c x2d_p22_cn x2d_p22_e x2d_p22_en x2d_p22_o x2d_p22_on x2d_p23_c x2d_p23_cn x2d_p23_e x2d_p23_en x2d_p23_o x2d_p23_on x2d_p24_c x2d_p24_cn x2d_p24_e x2d_p24_en x2d_p24_o x2d_p24_on x2d_p25_c x2d_p25_cn x2d_p25_e x2d_p25_en x2d_p25_o x2d_p25_on x2d_p26_c x2d_p26_cn x2d_p26_e x2d_p26_en x2d_p26_o x2d_p26_on x2d_p27_c x2d_p27_cn x2d_p27_e x2d_p27_en x2d_p27_o x2d_p27_on x2d_p28_c x2d_p28_cn x2d_p28_e x2d_p28_en x2d_p28_o x2d_p28_on x2d_p29_c x2d_p29_cn x2d_p29_e x2d_p29_en x2d_p29_o x2d_p29_on x2d_p2_c x2d_p2_cn x2d_p2_e x2d_p2_en x2d_p2_o x2d_p2_on x2d_p30_c x2d_p30_cn
Pin
AE13 AN12 AM13 AJ12 AL11 AG10 AF11 AN08 AM09 AL10 AL09 AD11 AG09 AN04 AM05 AL08 AJ09 AE04 AD05 AG02 AH01 AE06 AC09 AC05 AC04 AC02 AD01 AC07 AB09 AB03 AA05 W02 Y01 AA08 AB07 Y07 AA06 W04 W03 W08 W09 P03 P05 N07 P09 N03 N04 L06 N09 M01 N02 M05 L03 J13 F11 B13 A12 C11 E12 K07 L08
Signal Name
x2d_p30_e x2d_p30_en x2d_p30_o x2d_p30_on x2d_p31_c x2d_p31_cn x2d_p31_e x2d_p31_en x2d_p31_o x2d_p31_on x2d_p3_c x2d_p3_cn x2d_p3_e x2d_p3_en x2d_p3_o x2d_p3_on x2d_p4_c x2d_p4_cn x2d_p4_e x2d_p4_en x2d_p4_o x2d_p4_on x2d_p5_c x2d_p5_cn x2d_p5_e x2d_p5_en x2d_p5_o x2d_p5_on x2d_p6_c x2d_p6_cn x2d_p6_e x2d_p6_en x2d_p6_o x2d_p6_on x2d_p7_c x2d_p7_cn x2d_p7_e x2d_p7_en x2d_p7_o x2d_p7_on x2d_p8_c x2d_p8_cn x2d_p8_e x2d_p8_en x2d_p8_o x2d_p8_on x2d_p9_c x2d_p9_cn x2d_p9_e x2d_p9_en x2d_p9_o x2d_p9_on
Pin
H01 J02 K03 J03 L10 J07 D01 E02 H03 J05 E14 C14 J14 G13 D13 C13 F21 G20 C19 D19 J19 H19 E21 C22 A20 B19 G22 H21 D23 E23 A24 B23 J22 G23 E24 D25 A28 B27 J23 F25 J27 L24 E32 D33 J29 H31 L26 K27 J32 H33 J31 K31
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
248
RELEASED Data Sheet
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
PMC-2000164
ISSUE 3
ENHANCED TT1TM SWITCH FABRIC
4.6.2
Package Dimensions
This section outlines the mechanical dimensions for the Crossbar device. The package is a 1088 ceramic column grid array (CCGA). NOTE: Drawings are not to scale.
Figure 63. Crossbar CCGA Package Dimensions - Top and Side Views
D D1
E1
E
Top View A Pin A01 corner A1
A2
aaa
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
249
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 64. Crossbar CCGA package Dimensions - Bottom View
b
e
AN AM AL AK AJ AH AG AF AE AD AC AB AA Y W V U T R P N M L K J H G F E D C B A 02 04 03 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 01
Pin A01 corner
Bottom View
Table 34.
Crossbar CCGA Mechanical Specifications
e = 1.27 mm pitch Symbol Minimum A A1 A2 aaa D D1 E E1 M N b 0.48 NOTE: Column alloy is 90/10 PbSn Nominal 6.83 4.62 2.21 0.15 42.50 40.64 42.50 40.64 33 x 33 1088 0.52 mm Maximum mm mm mm mm mm mm mm mm Units
250
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
PMC-2000164
ISSUE 3
ENHANCED TT1TM SWITCH FABRIC
4.6.3
Part Number
The PMC-Sierra Part Number for the Crossbar is: Crossbar PM9312-UC
PMC Logo
0 9K0 37 2 P Q PM9312-UC Cr o ss b a r R ev A BXXLLLLLLLL
Country of Origin
Source "B" Part Number PMC Part Number Device Name Revision Lot Number Date Code
BYYW W X XX X XX X XX X XX
Terminal A01 Identifier
PM 9 3 12 - U C
PMC prefix IRD prefix IC Chipset family code, 1=TT1, 5 =TTX Sequential part number (1=SCH, 2=XBAR, 3=DS, 5=EPP) Temp. range (C = Commercial 0-70 degrees C) Package code (U = CCGA, H = CBGA)
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
251
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
252
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
5
Scheduler
This chapter contains information on the Scheduler device, part number PM9311-UC, available from PMC-Sierra, Inc. The Scheduler communicates directly with all of the fabric ports. Its purpose is to arbitrate the flow of cells through the fabric from the input ports, through the fabric, to the required output ports. The Scheduler has information about all cells that are waiting in queues at the input ports. The output ports tell the Scheduler when they can accept new cells from the input ports. At every cell time, the Scheduler uses all of this information to arbitrate between all of the inputs to establish the set of cells that can be transferred through the crossbar fabric to the output queues. The effectiveness of the Scheduler at performing this arbitration determines the throughput of the fabric under any given cell load. The Scheduler must have accurate, up to date information in order to arbitrate efficiently and fairly. Consequently, the EPPs send new state information to the Scheduler at every cell time.
5.1
BLOCK STRUCTURE
The block diagram of the Scheduler is shown in Figure 65 on page 255.
5.1.1
Port Block
The Scheduler can arbitrate between as many as 32 ports of OC-192c or 128 ports of OC-48c. The state associated with each port is maintained in a port block. Each port block contains a serial link, a port control unit, and some queue information memory. The serial link provides a full duplex, 1.6 Gbit/s, communications channel to the appropriate EPP. This channel is used to send new request and backpressure information to the Scheduler, and to send grant and routing tag information to the EPP. The port control unit provides a set of registers that indicate the current status of many elements of the port block, and that can also be used to control the behaviour of the port block. It is described further in Section 5.2 "Port State". The port block also has some local memory which contains the number of outstanding requests for every queue associated with this port.
5.1.2
Arbiter Block
The arbiter receives information from each of the port blocks. This information indicates which of the virtual output queues in each port have cells waiting to be forwarded to their appropriate destination port. At every cell time, the arbiter combines these inputs to decide which ports can forward a cell at some fixed time in the future. The resulting set of grants is then passed back to the port blocks.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
253
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
The arbiter has a number of control registers which are used primarily for diagnostics. Except for initial configuration after reset, the arbiter does not require any intervention from the OOB CPU.
5.1.3
OOB Interface and Control/Status Registers
All of the devices have an OOB interface. This interface allows a single local CPU to control and monitor all of the devices within a core fabric. Internally, each device provides registers that can be mapped into the CPU's address space. These registers are described in more detail in Section 5.4 "Scheduler Registers".
5.1.4
TDM Service
At every cell time the Scheduler may receive a TDM request from each port that intends to transfer a TDM cell (according to the TDM tables in the Port Processor). If a TDM request is received then the Scheduler will return a TDM Grant to that port, and that port will not be used in the arbitration of the best-effort traffic for that cell time. The Scheduler may also receive notification that a port is expecting to receive a TDM cell. This notification identifies the port that will source the TDM cell. If the Scheduler receives a TDM request from the designated source port then the destination port will receive an appropriate routing tag. However, if the source port has not made a TDM request then the destination port will be eligible to receive a best-effort cell. The Scheduler is the point of synchronization for the TDM service provided by the core fabric. The TDM control block generates synchronization signals at regular intervals. The period of these signals is determined either by internal registers, or by signals received from an EPP. The Scheduler sends the synchronization pulses to all 32 EPPs at the same time to indicate the start of a new TDM frame. See Section 1.2.5 "TDM Service" on page 22 for more information on the TDM service.
254
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 65. Scheduler Block Diagram
Port 0 Queue Information
requests p2s_p[0] s2p_p[0]
Serial Link
Port Control grants
Arbiter Port 31 Queue Information
requests p2s_p[31] s2p_p[31]
Serial Link
Port Control grants
core clocks ref_clk(n) plllock
PLL
TDM Control
OOB Interface Control/Status Registers
JTAG
oob_clk
oob_valid_L
oob_wait_L
oob_int_lo
oob_int_hi
oob_devsel[3:0]
soc_in(n)
oob_ad[7:0]
soc_out
jtag_tms
jtag_tck
pwrup_reset_in_L
jtag_trst_L
jtag_tdo
jtag_tdi
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
255
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
5.2
PORT STATE
The Scheduler controls all cell flow within the fabric for both normal, best-effort cells and TDM cells. The Scheduler can enable and disable cell flow on a port-by-port and class-of-service basis. When a new port is added to the system, it can be tested offline without affecting existing traffic flows. Each Scheduler port has a very simple state machine, as shown in Figure 66. The port's current state determines the type of cells it can, and cannot, forward through the fabric.
Figure 66. Port State Machine
reset or link is not Ready Disabled
SNBPRT[port] = 1 AND link is Ready
SNBPRT[port] = 0 OR link is not Ready
Enabled
Refresh cell Refresh cell
Ordinary cell Refresh_done cell
Refresh_done cell Refreshing Refreshed
After reset, the port is in the disabled state. No cells will flow to or from this port when it is disabled. Once the enable non-TDM traffic register for this port is set to 1, then the port is enabled. While the port is enabled, it receives requests from the EPP and issues requests to the arbiter. It receives grants from the arbiter and passes those grants to the EPP. It also receives routing tags that it passes on to the EPP. At any time if errors are received on the appropriate serial link then the port will be returned to the disabled state and must be reset before it can be enabled. The CPU initiates a Scheduler refresh by issuing a "Go Refresh" command to the EPP. The EPP then sends 384 Refresh cells to the Scheduler. The Scheduler port transitions to the refreshing state as soon as it receives the first of these Refresh cells. Once the EPP has sent all of the Refresh cells it then sends Refresh_Done cells, causing the Scheduler port to transition to the refreshed state. The CPU then terminates the refresh operation by issuing a "Go Scheduler" command to the EPP. This causes the EPP to send ordinary cells. As soon as the Scheduler port receives an ordinary cell it will transition back to the enabled state.
256
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
TDM traffic operates in two of the three active states enabled and refreshing, provided the corresponding bit in the enable_TDM_traffic register is set to 1. TDM traffic does not operate if the port is disabled. TDM traffic and non-TDM traffic are enabled independently. NOTE: TDM traffic does not operate while the Scheduler is in the refreshed state. So the CPU should try to minimize the time that the Scheduler is in the refreshed state if TDM traffic is used. Non-TDM traffic only operates in the enabled state. When the port is in refreshing or refreshed states, non-TDM traffic does not operate for this port: requests are not made and backpressure is asserted to prevent other ports from sending to this port.
Table 35. Non-TDM Traffic
Scheduler Port State Disabled Enabled Enabled Refreshing Refreshed
Enable non-TDM Register don't care 1 (non-TDM enabled) 0 (non-TDM disabled) Don't care Don't care
Unicast Backpressure Asserted Not used in ETT1 Asserted Asserted Asserted
Multicast Backpressure De-asserted As per information from EPP Asserted Asserted Asserted
Table 35 shows whether non-TDM traffic can flow, based on the port state. NOTE: Multicast backpressure is not asserted if the port is disabled. This is to prevent head of line blocking in the case where one port is sending a multicast cell to a port which is disabled. See Section 1.3.4 "Multicast Traffic (subport mode)" on page 46 for more on the multicast blocking capability.
5.3
FAULT TOLERANCE
The Scheduler reacts to link faults in different ways, depending upon the mode it is in. The mode is set by the CRC_Action bit in the control register. This bit is set only if the Scheduler is the primary scheduler in a dual scheduler system, otherwise it is not set. In both modes, if a Scheduler port detects a CRC error on the incoming link from the EPP, or the link goes down (the Ready line transitions from active to inactive for at least one clock cycle), then the port does the following: * The Scheduler port is placed into the disabled state NOTE: The Enable Port register will not reflect this change.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
257
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
*
The appropriate register (CRC or Link Ready Inactive) is updated.
This port asserts backpressure to all other ports. Once the CPU determines that this port can continue operation, then a refresh operation must be performed on this port and then the port can be enabled. If the CRC_Action bit is set, then in addition to the above actions, the Scheduler also asserts reset to all of its serial links to all EPPs. This turns off all of the links, forcing all of the EPPs to switch to the secondary Scheduler. All of the primary Scheduler's ports are set to the disabled state and the enable_port register is set to 0. The Scheduler must be reset by the CPU and have its queue information refreshed before it is re-enabled.
258
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
5.4
SCHEDULER REGISTERS
The Scheduler device select low-order bits are hard wired on the board by four pins (oobdev_sel[3:0]) on the Scheduler package. See Section 1.7.1.3 "Individual Device Selects" on page 79 for more information.
5.4.1
Summary
The following table is a summary of information for all registers in the Scheduler. See the following Descriptions section for more information on individual registers. Read and Clear means that reading the register causes it to be cleared (reset to zero)
Table 36. Scheduler Register Summary
Address 00000h 00004h 00008h 0000Ch 00010h 00014h 00018h 0001Ch 00020h 00024h 00028h 00030h 00034h 00038h 0003C 00040h
Access Read Only Read/Write Read/Write Read/Write Read and Clear Read/Write Read and Clear Read/Write Read and Clear Read/Write Read and Clear Read/Write Read Only Read/Write Read/Write Read/Write Status
Register
Symbol SSTS SCTRLRS SIRLMSK SIRHMSK SIR SCRCMSK SCRC SRDYDMSK SRDYDN SRDYUMSK SRDYUP SAIBRS SAIBRDY SENBPRT SPORTEN FCCSYN
Default Value 00000000h 00000130h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h FFFFFFFFh 00000000h 00000000h 00000000h 00000000h
Control and Reset Low Priority Mask High Priority Mask Interrupt Register CRC Error Interrupt Mask CRC Errors Link Ready Inactive Interrupt Mask Link Ready Inactive Link Ready Active Interrupt Mask Link Ready Active AIB Reset AIB Ready Enable Port AIB Tx Enable Flow Control Crossbar Sync
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
259
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 36. Scheduler Register Summary
Address 00080h 00084h 00088h 0008Ch 00090h 00094h 00098h 0009Ch 00100h
Access Read/Write Read/Write Read/Write Read/Write Read Only Read Only Read Only Read/Write Read/Write
Register Enable non-TDM Traffic Enable TDM Traffic Reset Port Freeze Port Port is Refreshing Port is Refreshed TDM Status TDM Control PLL control/status
Symbol SNTDMEN STDMEN SPORTRS SPORTFRZ SRFRSNG SRFRS STDMSTS STDMCTRL SPLL
Default Value 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 00000000h 0001447Ch
260
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
5.4.2
Scheduler Register Descriptions
Read and Clear means that reading the register causes it to be cleared (reset to zero). All bits labled as Reserved should be set to 0.
5.4.2.1 Symbol:
Status SSTS
Address Offset: 00000h Default Value: 30000000h Access: Read Only Status register. Bits 31:28 27:24 23:2 1 0 Chip Revision Number. Reserved. High Priority Interrupt. 1 = an outstanding high priority interrupt. One of the bits in the Interrupt Register is set, and is enabled via its corresponding high priority mask. Low Priority Interrupt. 1 = an outstanding low priority interrupt. One of the bits in the Interrupt Register is set, and is enabled via its corresponding low priority mask. Description Chip ID Number. Identifies the specific device.
5.4.2.2 Symbol:
Control and Reset SCTRLRS
Address Offset: 00004h Default Value: 00000130h Access: Read/Write Control and Reset register. Bits 31-9 8-4 3 Reserved. BP_FIFO. Specifies the depth of the backpressure FIFO. This is an internal FIFO and users should set these bits to a value of 14h after reset. The default value is 13h Reserved. Description
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
261
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Bits
Description (Continued) CRC Action. This specifies what action is to be taken when an enabled port (port enable register bit is 1) encounters a CRC error from its attached Port Processor. If set to 0 then the port with the error is shut down and the Scheduler continues to arbitrate among the remaining ports. If set to 1 then the Scheduler will reset all serial links, effectively disabling all 32 ports. This bit should only be set to 1 for the primary Scheduler within a core with redundant Schedulers. Subport mode. If set to 1 then the Scheduler operates in subport mode, in which each port is considered to be four subports with only one priority. This bit is not used in the ETT1 Scheduler and must be set to 0. Reset. Writing a 1 to this location will reset the entire device. It is equivalent to a hardware reset. This register is cleared automatically when the chip is reset. This bit will always read as 0 and it is not necessary to write a 0 having just written a 1. Soft reset takes 1mS to complete.
2
1
0
5.4.2.3 Symbol:
Low Priority Mask SIRLMSK
Address Offset: 00008h Default Value: 00000000h Access: Read/Write Interrupt Mask for interrupts. Bits 31:5 4:0 Reserved. Low Priority Mask. Mask bits for low priority interrupts. Each mask bit is set to 1 to enable a low priority interrupt when the corresponding bit in the Status Register is 1. Description
5.4.2.4 Symbol:
High Priority Mask SIRHMSK
Address Offset: 0000Ch Default Value: 00000000h Access: Read/Write Interrupt Mask for interrupts. Bits 31:5 Reserved. Description
262
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Bits 4:0
Description High Priority Mask. Mask bits for high priority interrupts. Each mask bit is set to 1 to enable a high priority interrupt when the corresponding bit in the Status Register is 1.
5.4.2.5 Symbol:
Interrupt Register SIR
Address Offset: 00010h Default Value: 00000000h Access: Read and Clear Interrupt Register. Bits 31:5 4 Reserved. All Ports Refreshed. During a refresh operation, this bit will be set when all enabled ports have been refreshed. See also Refreshed Status register. This bit is cleared on reading this register. TDM Sync Lost. This bit is set if the TDM feature has been configured, and no Sync signal is received for three consecutive sync periods. This would suggest that a failure has occurred in either the Port Processor or its attached linecard. This bit is cleared on reading this register. Ready Inactive. This is the logical OR of the Link Ready Inactive Interrupt register, after it has been masked by the Link Ready Inactive Interrupt Mask register. This bit is set if a serial link goes from active to inactive and the corresponding mask bit is 1. This bit is not cleared when read - the user must clear the Link Ready Inactive Interrupt register. Ready Active. This is the logical OR of the Ready Active interrupt register, after it has been masked. This is set if a serial link goes from inactive to active and the corresponding mask bit is 1. This bit is not cleared when read - the user must clear the Ready Active interrupt register. CRC Error. This is the logical OR of the CRC interrupt register, after it has been masked. This is set if a CRC error occurs and the corresponding mask bit is 1. This bit is not cleared when read - you must clear the CRC register. Description
3
2
1
0
5.4.2.6 Symbol:
CRC Error Interrupt Mask SCRCMSK
Address Offset: 00014h Default Value: 00000000h Access: Read/Write
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
263
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Interrupt mask for CRC errors. Bits 31:0 Description CRC Error Interrupt Mask. Each bit is used to mask (enable) interrupts when the corresponding bit in the CRC Error register is set to 1. The interrupt is enabled when the bit is 1.
5.4.2.7 Symbol:
CRC Errors SCRC
Address Offset: 00018h Default Value: 00000000h Access: Read and Clear CRC Interrupt Error Register Bits 31:0 Description CRC Errors. Each bit indicates if a CRC error has occurred on the appropriate link (Port Processor to Scheduler) since the last time this register was read. Mask off unwanted (unused ports) bits via the CRC Error Interrupt Mask.
5.4.2.8 Symbol:
Link Ready Inactive Interrupt Mask SRDYDMSK
Address Offset: 0001Ch Default Value: 00000000h Access: Read/Write Enables interrupts when the corresponding bit in the Link Ready Inactive register is set to 1. Bits 31:0 Description Link Ready Inactive Interrupt Mask. Each bit is used to mask (enable) interrupts when the corresponding bit in the Link Ready Inactive register is set to 1. The interrupt is enabled when the bit is 1.
264
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
5.4.2.9 Symbol:
Link Ready Inactive SRDYDN
Address Offset: 00020h Default Value: 00000000h Access: Read and Clear Each bit indicates if the Ready signal from a link has transitioned from Active to Inactive (the link has gone down for some reason). NOTE: The link may be up again (active) by the time the CPU reads this register. Mask off unwanted (unused ports) bits via the Link Ready Inactive Interrupt Mask. Reading this register will clear all bits. Bits 31:0 Description Link Ready Inactive. Each bit indicates if the Ready signal from a link has transitioned from Active to Inactive.
5.4.2.10 Symbol:
Link Ready Active Interrupt Mask SRDYUMSK
Address Offset: 00024h Default Value: 00000000h Access: Read/Write Enables interrupts when the corresponding bit in the Link Ready Active register is set to 1. Bits 31:0 Description Link Ready Active Interrupt Mask. Each bit is used to mask (enable) interrupts when the corresponding bit in the Link Ready Active register is set to 1. The interrupt is enabled when the bit is 1.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
265
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
5.4.2.11 Symbol:
Link Ready Active SRDYUP
Address Offset: 00028h Default Value: 00000000h Access: Read and Clear Each bit indicates if the Ready signal from a link has transitioned from Inactive to Active (the link has come up). NOTE: The link may be down again (inactive) by the time the CPU reads this register. Mask off unwanted (unused ports) bits via the Link Ready Active Interrupt Mask. Reading this register will clear all bits. Bits 31:0 Description Link Ready Active. Each bit indicates if the Ready signal from a link has transitioned from Inactive to Active.
5.4.2.12 Symbol:
AIB Reset SAIBRS
Address Offset: 00030h Default Value: FFFFFFFFh Access: Read/Write Used to assert reset to the AIB link for each port. Bits 31:0 Description AIB Reset. Each bit is used to assert reset to the AIB link for each port. Reset is asserted when the bit is 1. If reset the link will not operate or transition to ready. This register is set to 0xFFFF_FFFF on power-up reset or if the reset bit in the Control Register is asserted.
266
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
5.4.2.13 Symbol:
AIB Ready SAIBRDY
Address Offset: 00034h Default Value: 00000000h Access: Read Only Indicates if the corresponding AIB link is ready. Bits 31:0 Description AIB Ready. Each bit indicates if the corresponding AIB link is ready (1 = active) or not (0 = inactive).
5.4.2.14 Symbol:
Enable Port SENBPRT
Address Offset: 00038h Default Value: 00000000h Access: Read/Write This register is used to enable a scheduler port to carry traffic. After reset this register is 0 - all scheduler ports are disabled. While a scheduler port is disabled it will not respond to requests from the attached port card, nor will it issue grants. Bits 31:0 Description Enable Port. Each bit indicates if the corresponding port should be enabled. NOTE: Reading this register only reflects the last value written to it. A CRC error, for example, might cause a Scheduler port to become inactive, but that change will not be reflected in this register. Similarly, if a port is active, then setting the port to inactive by writing to the Reset Port register will not be reflected in this register even though the port will be inactive. Whenever a port is reset the appropriate bit in this register should also be cleared.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
267
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
5.4.2.15 Symbol:
AIB Tx Enable SPORTEN
Address Offset: 0003Ch Default Value: 00000000h Access: Read/Write This is used to enable the transmitter of the corresponding AIB link. Bits 31:0 Description AIB Tx Enable. If the corresponding port is not physically present in the system then the appropriate bit should be set to zero to reduce unwanted electrical noise and to minimize unwanted effects when the port board is inserted
5.4.2.16 Symbol:
Flow Control Crossbar Sync FCCSYN
Address Offset: 00040h Default Value: 00000000h Access: Read/Write NOTE: This register is only used in conjunction with the Enhanced Port Processor. It is not used with the Port Processor. The counter is reloaded at reset or whenever a refresh_go occurs. Thus, the counter can be synchronized between two Scheduler devices. To force the counter to reload, set the Run/Stop bit to 0, write a new count value to the Synch_period, and then set the run/stop bit to 1. Dual Schedulers should always be refreshed (synchronized) after this register has been modified. Bits 31:16 15:1 Description Synch_period. This value denotes the period at which OC48 synch pulses will be issued. Setting this to the recommended value of 32 means that the OC48_synch bit will be set to 1 for every one in 32 frames sent to the Port Processor. Reserved. Run/Stop. This bit enables the OC-48 Synch counter. When 0 the counter does not run and is continuously reloaded with the value in the synch_period field. When this bit is set to 1 the counter counts down by one for each cell time. When the counter reaches the value one a synchronization pulse is issued in the scheduler to Port Processor frame on all ports (a bit is set in the frame).
0
268
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
5.4.2.17 Symbol:
Enable non-TDM Traffic SNTDMEN
Address Offset: 00080h Default Value: 00000000h Access: Read/Write Indicates if the corresponding port can send and receive non-TDM traffic. Bits 31:0 Description Enable non-TDM Traffic. Each bit indicates if the corresponding port can send and receive non-TDM traffic (i.e. best-effort cells of priority 0,1,2 or 3). This bit should be de-asserted (set to 0) before telling the EPP to refresh the Scheduler's state.
5.4.2.18 Symbol:
Enable TDM Traffic STDMEN
Address Offset: 00084h Default Value: 00000000h Access: Read/Write
Indicates if the corresponding port can send and receive TDM traffic.
Bits 31:0
Description Enable TDM Traffic. Each bit indicates if the corresponding port can send and receive TDM traffic (bit set to 1). Each bit should be 0 if the appropriate port does not send or receive TDM traffic.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
269
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
5.4.2.19 Symbol:
Reset Port SPORTRS
Address Offset: 00088h Default Value: 00000000h Access: Read/Write Resets each port. Bits Description Reset Port. Each bit indicates if the corresponding port is held in reset (1). By asserting and then de-asserting this bit for a port, the internal state associated with that port is reset so that the port is in the Disabled state (see Section 5.2 on page 256). This register should not be left asserted. Do not use this register to keep ports in the inactive state. Instead, use the SENBPRT register to enable or disable ports.
31:0
5.4.2.20 Symbol:
Freeze Port SPORTFRZ
Address Offset: 0008Ch Default Value: 00000000h Access: Read/Write Freezes Scheduler ports. Bits 31:1 0 Reserved. Freeze Port. This is used for diagnostic purposes only! Asserting this bit will cause all ports to halt operation and to send a freeze command to their attached Port Processor. De-asserting this bit will allow traffic flow to be resumed. Caution: Cell loss may occur when modifying this bit. Description
5.4.2.21 Symbol:
Port is Refreshing SRFRSNG
Address Offset: 00090h Default Value: 00000000h Access: Read Only
270
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Refreshing status. Bits 31:0 Description Port is Refreshing. During a refresh operation the port transitions from active to refreshing when it receives a refresh cell from the Port Processor. This bit is 1 if the port is currently refreshing (has received a refresh cell) but has not yet completed the refresh operation.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
271
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
5.4.2.22 Symbol:
Port is Refreshed SRFRS
Address Offset: 00094h Default Value: 00000000h Access: Read Only Refreshed status. Bits Description Port is Refreshed. During a refresh operation the port transitions from active to refreshing when it receives a refresh cell from the Port Processor. Once all refresh cells have been received the port is considered to be "refreshed". This bit is 1 if the port is currently refreshed i.e. it has been refreshed but has not yet restarted normal operation. The refreshed bit in the Interrupt Register is set when all enabled ports have this bit set to one, and at least one port is enabled.
31:0
5.4.2.23 Symbol:
TDM Status STDMSTS
Address Offset: 00098h Default Value: 00000000h Access: Read Only Indicates TDM status. Bits 31:17 16 15:11 10:0 Reserved. Indicates the TDM frame (0 or 1) that was received in the most recent cell from the current port that is providing the suggested TDM sync. Reserved. The current period minus one at which TDM sync signals will be issued to the Port Processors. So if the current TDM Synch is being generated every 512 cell times then this field will be read as 511. Description
5.4.2.24 Symbol:
TDM Control STDMCTRL
Address Offset: 0009Ch
272
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Default Value: 00000000h Access: Read/Write TDM Control Register Bits 31 Description Enable TDM Sync. Set this bit to 1 to enable TDM sync to be sent out to all the Port Processors. Default is 0 (TDM sync is off). This bit only controls the propagation of TDM Synch out of the Scheduler, and does not affect the internal TDM PLL. Load Initial Period. Set this bit to 1 and then 0 to cause the Scheduler to use the Initial Period field in this register as the current TDM sync period. (Only used if the Scheduler generates TDM Synch itself) Reserved. Initial Period. This can be used in conjunction with the Load Initial Period bit to set an initial TDM period. Only use this if there is not a currently valid port supplying TDM sync and TDM frame. (Only used if the Scheduler generates TDM Synch itself). This value is the initial period minus one at which TDM sync signals will be issued to the Enhanced Port Processors Reserved. FIFO Depth. Leave this value set to zero. Transmitted TDM Frame (0 or 1). This is the value of the TDM Table select that is sent to the PPs when bit 6 is 0. (Only used if the Scheduler generates TDM Synch itself) Current Port Valid. Indicates that the Current Port field is valid, i.e. there is a port that is currently sending correct TDM sync and frame information to the Scheduler. This bit is 0 if the Scheduler generates TDM Synch itself. Reserved. Current Port. Specifies the port which is currently sending suggested TDM sync signal and TDM frame (which is echoed back out to all Port Processors). This field is irrelevant unless bit 6 is set. If these bits are modified then bit 6 should be set to 0 and then back to 1.
30 29:27 26:16 15:11 10:8 7 6 5 4:0
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
273
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
5.4.2.25 Symbol:
PLL Control/Status SPLL
Address Offset: 00100h Default Value: 0001447Ch Access: Read/Write Controls operation of the internal PLL (Phase locked loop). After a power on reset the PLL itself is held in reset. This is reflected in bit 16 of this register. The local CPU must reset this bit to 0 to enable operation of the device, and thus should write 0000447Ch to this register. Bit 31:17 16 15:0 Description PLL status. These bits reflect internal PLL operation status and should be ignored. Reset PLL. When set to 1 the PLL is held reset. The supplied reference clock will be used as the internal clock. The serial links will not be operational. This bit will be 1 after power-up reset and should be deasserted for normal operation. PLL reset takes 10mS to complete. PLL control.
274
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
5.4.3
Input Queue Memory (IQM)
The Scheduler has internal memory which stores the number of waiting cells at every input port. This memory can be read by the CPU. Caution: while the memory can also be modified, doing so may result in cell loss. CPU access to this memory is provided for diagnostic purposes only. After reset the memory is cleared to zero. Each Scheduler port (0:31) has four IQM units, one for each best-effort priority. Each IQM is organized as two independent memories: the first is 64 words of 32 bits and stores multicast fanout information. The second is 32 words of 7 bits and stores the number of waiting cells at each unicast virtual output queue. The individual entries in each IQM are addressed by the CPU as shown below:
19 0 0 1 15 Port 11 Pri 0 M/U 7 Addr/QID 3 0 0 0
Port is the port number 0:31. Pri is the selected best effort priority 0:3. M/U is 1 for the multicast fanout memory and 0 for the unicast request count memory. Addr/QID is the multicast fanout address or a unicast VOQ identifier (only 0:31). So, for example, unicast VOQ 10 contains the number of requests (cells) that are waiting to go from this port to output port 10. The multicast fanout addresses are organized as a circular list. However the head of the list cannot be determined by the CPU except immediately after a refresh operation, at which time the second entry in the multicast fanout queue is at location zero. The first entry is held in a private register and is not visible to the CPU. Although the IQM should not be modified during normal operation, it is prudent to verify the integrity of the memory after the Scheduler is reset.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
275
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
5.5
SCHEDULER SIGNAL DESCRIPTIONS
This section describes the Scheduler signals. The following notation is used to describe the signal type: I O B Input pin Output pin Bidirectional Input/Output pin
The signal description also includes the type of buffer used for the particular signal: PECL STI PECL are Pseudo-ECL (positive voltage ECL) compatible signals. STI is a very high-speed asynchronous communications protocol used to connect devices that may be 1 to 15 meters apart. All HSTL specifications are consistent with EIA/JEDEC Standard, EIA/JESD8-6 "High Speed Transceivers Logic (HSTL): A 1.5V Output Buffer Supply Voltage based Interface Standard for Digital Integrated Circuits", dated 8/95. Refer to these specifications for more information. The CMOS Buffers are either 3.3 V compatible or 2.5 V Low Voltage TTL compatible signals, as noted.
HSTL
CMOS
Table 37.
Scheduler Signal Descriptions
Name
I/O
Type OOB Interface
Description
oob_ad[7:0] oob_clk oob_devsel0 oob_devsel1 oob_devsel2 oob_devsel3 oob_int_hi oob_int_lo
B I I I I I O O
CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS
OOB Address bus OOB CLock OOB device select OOB device select OOB device select OOB device select OOB Interrupt OOB Interrupt
276
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 37.
Scheduler Signal Descriptions (Continued)
Name oob_valid_L oob_wait_L
I/O I O
Type CMOS CMOS OOB Control OOB Control
Description
AIB Interface p2s_p[31:0]_[c, cn, e, en, o, on] s2p_p[31:0]_[c, cn, e, en, o, on] I O STI STI Port Processor to Scheduler AIB Scheduler to Port Processor AIB
Power Supply, Clock Source, Reset, And Diagnostics plllock pwrup_reset_in_L soc_out VDDA VDD GND ref_clk and ref_clkn soc_in and soc_inn I I O I O CMOS CMOS CMOS Isolated Supply Supply Supply PECL PECL Indicates if the PLL has locked (1) Synchronous, Active Low Reset Buffered soc_in (NC) VDD (2.5 V) for PLL VDD (2.5 V) GND (0 V) System 200MHz clock (differential) System Start-of-cell (differential)
JTAG Interface jtag_tck jtag_tdi jtag_tdo jtag_tms jtag_trst_L I I O I I CMOS_ 2.5_only CMOS_ 2.5_only CMOS_ 2.5_only CMOS_ 2.5_only CMOS_ 2.5_only 1149.1 JTAG 2.5V ONLY! 1149.1 JTAG 2.5V ONLY! 1149.1 JTAG 2.5V ONLY! 1149.1 JTAG 2.5V ONLY! 1149.1 JTAG 2.5V ONLY!
ASIC Manufacturing Test Interface
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
277
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 37.
Scheduler Signal Descriptions (Continued)
Name ce0_io ce0_scan ce0_testm3 lssd_ce1_a lssd_ce1_b lssd_ce1_c1 lssd_ce1_c2 lssd_ce1_c3 lssd_scan_in[15:0] lssd_scan_out[15:0] mon[15:0] plltest_in plltest_out test_di1
I/O I I I I I I I I I O O I O I
Type CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS CMOS Test (GND) Test (GND) Test (GND) Test (VDD) Test (VDD) Test (VDD) Test (VDD) Test (VDD)
Description
Scan input (GND) Scan output (NC) Test (NC) Test (GND) Test output (NC) Test (VDD) Should be driven to GND during reset. All outputs are tristated when low. Test (VDD) Should be driven to GND during reset. All outputs are tristated when low. Test (VDD) Test (VDD)
test_di2 test_lt test_ri
I I I
CMOS CMOS CMOS
278
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
5.6 5.6.1
PINOUT AND PACKAGE INFORMATION Pinout Tables
Table 38. Scheduler Pinout (left side)
01 A B C D E F G H J K L M N P R T U V W Y AA AB AC AD AE AF AG AH AJ AK AL AM AN
No Pin oob_ad0 jtag_tms oob_wait_L VDDA p2s_p16_c
02
pwrup_ reset_in_L GND oob_ad1 VDD oob_int_lo GND
03
test_di2 soc_out jtag_tck oob_ad2 plltest_out s2p_p16_on plltest_in p2s_p17_cn
04
mon4 VDD soc_in GND oob_ad3 VDD s2p_p16_o GND
05
ref_clk mon5 ref_clkn soc_inn plllock oob_ad6 oob_ad7 p2s_p16_cn
06
p2s_p0_cn GND s2p_p0_o VDD mon2 GND oob_ad4 VDD
07
jtag_trst_L s2p_p1_o ce0_testm3 s2p_p0_on mon3 mon1 mon0 oob_ad5
08
s2p_p0_en VDD p2s_p1_c GND p2s_p0_c VDD mon7 GND oob_clk VDD
09
p2s_p0_o p2s_p0_on s2p_p0_e p2s_p1_cn s2p_p1_on s2p_p1_c p2s_p0_e mon6 mon9 oob_int_hi
10
s2p_p1_e GND
11
s2p_p2_en p2s_p1_o
s2p_p1_en p2s_p1_on VDD s2p_p2_o GND p2s_p1_en VDD p2s_p2_cn p2s_p2_c s2p_p2_on p2s_p2_e s2p_p2_cn
UNUSED s2p_p17_on s2p_p16_e VDD
p2s_p16_on p2s_p16_o s2p_p16_en p2s_p17_c s2p_p17_o s2p_p17_c p2s_p16_e s2p_p17_en GND s2p_p17_e VDD s2p_p18_on GND s2p_p17_cn
s2p_p0_cn s2p_p1_cn GND s2p_p0_c mon8 s2p_p16_cn
s2p_p18_e p2s_p17_on p2s_p17_o p2s_p18_c p2s_p18_cn s2p_p18_o s2p_p18_cn p2s_p17_en p2s_p16_en s2p_p16_c p2s_p18_o VDD s2p_p18_en GND s2p_p19_on VDD s2p_p19_cn GND s2p_p18_c VDD
s2p_p19_en s2p_p19_e p2s_p18_on p2s_p19_c p2s_p19_cn s2p_p19_o p2s_p19_en p2s_p19_e s2p_p19_c p2s_p18_e p2s_p17_e p2s_p19_o GND UNUSED VDD UNUSED GND UNUSED GND UNUSED VDD UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED GND UNUSED VDD UNUSED VDD UNUSED GND UNUSED UNUSED lssd_scan_ in15 UNUSED lssd_scan_ out14 UNUSED UNUSED VDD UNUSED GND UNUSED GND UNUSED VDD UNUSED UNUSED lssd_scan_ out15 oob_ devsel0 lssd_scan_ in14 UNUSED UNUSED GND UNUSED VDD UNUSED VDD UNUSED GND p2s_p18_en UNUSED UNUSED UNUSED UNUSED UNUSED s2p_p21_o
UNUSED p2s_p19_on UNUSED UNUSED UNUSED UNUSED UNUSED s2p_p20_en VDD UNUSED VDD s2p_p20_e GND UNUSED UNUSED UNUSED UNUSED UNUSED
p2s_p20_o p2s_p20_on s2p_p21_e s2p_p20_cn s2p_p20_c p2s_p20_en s2p_p20_o s2p_p20_on p2s_p20_cn s2p_p21_on s2p_p22_on s2p_p21_en VDD p2s_p21_o GND p2s_p20_e VDD p2s_p20_c GND p2s_p21_cn VDD p2s_p23_c
p2s_p21_on s2p_p22_e s2p_p22_en s2p_p21_cn s2p_p21_c p2s_p21_en p2s_p21_c s2p_p22_o s2p_p23_o p2s_p23_cn UNUSED p2s_p22_o GND p2s_p22_on VDD p2s_p21_e GND p2s_p22_c VDD UNUSED UNUSED UNUSED GND s2p_p24_cn
s2p_p23_e s2p_p23_en p2s_p23_o s2p_p22_cn p2s_p22_e p2s_p22_cn s2p_p23_on UNUSED p2s_p23_on VDD s2p_p22_c GND s2p_p23_c VDD UNUSED GND UNUSED VDD UNUSED UNUSED UNUSED UNUSED p2s_p24_cn GND
s2p_p24_c s2p_p25_c VDD s2p_p26_c
lssd_scan_ UNUSED p2s_p22_en p2s_p23_en UNUSED out13 s2p_p23_cn UNUSED UNUSED lssd_scan_ in12 UNUSED UNUSED GND UNUSED VDD UNUSED GND UNUSED p2s_p23_e lssd_scan_ in13 UNUSED lssd_scan_ in11 UNUSED lssd_scan_ out11 VDD UNUSED GND UNUSED VDD UNUSED UNUSED lssd_scan_ out12 UNUSED
UNUSED p2s_p24_en p2s_p25_e p2s_p26_en VDD s2p_p25_cn GND s2p_p27_cn
p2s_p24_c s2p_p25_o s2p_p26_o p2s_p26_cn GND s2p_p26_on VDD p2s_p26_c
lssd_scan_ lssd_scan_ p2s_p25_c s2p_p24_en s2p_p25_e p2s_p25_o s2p_p24_o in9 in10 s2p_p24_on GND p2s_p25_cn VDD p2s_p24_o GND p2s_p25_on lssd_scan_ lssd_scan_ s2p_p25_on s2p_p24_e p2s_p24_on s2p_p25_en s2p_p26_e out10 out9
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
279
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 39.
Scheduler Pinout (center)
12 A B C D E F G H J K L M N P R T U V W Y AA AB AC AD AE AF AG AH AJ AK AL AM AN
p2s_p2_on VDD s2p_p2_e GND s2p_p3_o VDD p2s_p3_on GND s2p_p2_c VDD p2s_p0_en GND oob_valid_L VDD oob_ devsel2 GND UNUSED GND UNUSED VDD UNUSED GND
13
s2p_p3_e s2p_p3_en p2s_p2_o s2p_p3_cn s2p_p3_c s2p_p3_on UNUSED p2s_p3_o mon15 p2s_p2_en p2s_p1_e mon11 VDD oob_ devsel1 UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED VDD UNUSED
14
p2s_p3_cn GND p2s_p3_en VDD p2s_p3_e GND UNUSED VDD UNUSED GND UNUSED VDD mon10 GND oob_ devsel3 VDD GND VDD UNUSED GND UNUSED VDD
15
UNUSED p2s_p3_c UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED mon12 mon13 mon14 GND UNUSED UNUSED UNUSED GND UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED p2s_p27_on UNUSED
16
UNUSED VDD UNUSED GND UNUSED VDD ce0_io GND ce0_scan VDD UNUSED GND UNUSED VDD UNUSED GND VDD GND UNUSED VDD UNUSED GND UNUSED VDD lssd_scan_ out8 GND lssd_scan_ in8 VDD UNUSED GND UNUSED VDD UNUSED
17
UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED GND UNUSED VDD GND VDD UNUSED GND UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED
18
UNUSED VDD UNUSED GND UNUSED VDD test_di1 GND test_ri VDD UNUSED GND UNUSED VDD UNUSED GND VDD GND UNUSED VDD UNUSED GND UNUSED VDD lssd_scan_ in7 GND lssd_scan_ out7 VDD UNUSED GND UNUSED VDD UNUSED
19
UNUSED s2p_p4_en UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED GND UNUSED UNUSED UNUSED GND UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED s2p_p28_e
20
s2p_p4_e GND UNUSED VDD UNUSED GND UNUSED VDD UNUSED GND p2s_p4_c VDD UNUSED GND UNUSED VDD GND VDD UNUSED GND UNUSED VDD
21
p2s_p4_on p2s_p4_o
22
s2p_p5_e VDD
s2p_p5_en p2s_p5_on s2p_p4_c s2p_p4_ cn p2s_p4_e UNUSED UNUSED s2p_p4_o s2p_p5_o s2p_p6_on UNUSED VDD UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED VDD UNUSED GND p2s_p4_en VDD s2p_p4_ on GND p2s_p5_cn VDD s2p_p7_o GND UNUSED VDD UNUSED GND UNUSED GND UNUSED VDD UNUSED GND
p2s_p24_e p2s_p25_en s2p_p27_c VDD p2s_p26_e GND UNUSED VDD UNUSED GND
p2s_p28_cn s2p_p30_o s2p_p31_on GND s2p_p29_on VDD
s2p_p26_cn p2s_p27_e GND UNUSED
UNUSED s2p_p28_on p2s_p29_c VDD UNUSED GND UNUSED VDD UNUSED GND UNUSED UNUSED p2s_p28_en GND s2p_p28_o VDD
p2s_p27_en UNUSED VDD s2p_p27_o
s2p_p27_on p2s_p27_cn UNUSED GND p2s_p27_c VDD
s2p_p28_c p2s_p28_e s2p_p28_cn GND
s2p_p26_en p2s_p26_on UNUSED VDD s2p_p27_e GND
s2p_p29_e p2s_p29_o p2s_p28_on VDD
p2s_p26_o s2p_p27_en p2s_p27_o
UNUSED s2p_p28_en p2s_p28_o s2p_p29_en
280
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 40.
Scheduler Pinout (right side)
23 A B C D E F G H J K L M N P R T U V W Y AA AB AC AD AE AF AG AH AJ AK AL AM AN
p2s_p5_o s2p_p6_en s2p_p6_e s2p_p5_c s2p_p5_cn p2s_p4_cn s2p_p5_on p2s_p5_c p2s_p6_c p2s_p7_cn UNUSED s2p_p8_c
24
GND p2s_p6_o VDD p2s_p5_e GND s2p_p6_o VDD p2s_p7_c GND s2p_p8_cn VDD
25
s2p_p7_e p2s_p7_on p2s_p5_en p2s_p6_e p2s_p6_cn s2p_p7_on UNUSED UNUSED UNUSED p2s_p8_e s2p_p10_cn
26
p2s_p7_o VDD s2p_p6_c GND s2p_p7_c VDD UNUSED GND UNUSED VDD p2s_p9_e GND
27
s2p_p6_cn test_lt s2p_p7_cn UNUSED UNUSED UNUSED UNUSED
28
GND p2s_p7_e VDD UNUSED GND UNUSED VDD
29
lssd_ce1_ c1 p2s_p7_en lssd_ce1_b UNUSED lssd_ce1_ c2 UNUSED UNUSED p2s_p8_c s2p_p9_o s2p_p10_o
30
UNUSED VDD UNUSED GND UNUSED VDD s2p_p8_on GND p2s_p9_cn VDD
31
lssd_ce1_ c3 UNUSED jtag_tdo UNUSED UNUSED s2p_p8_o UNUSED p2s_p9_c s2p_p8_e s2p_p9_en
32
UNUSED GND UNUSED VDD UNUSED GND s2p_p9_on VDD p2s_p8_on GND
33
UNUSED UNUSED jtag_tdi UNUSED UNUSED p2s_p8_cn UNUSED s2p_p8_en p2s_p8_o s2p_p9_e
p2s_p6_on s2p_p7_en
lssd_ce1_a p2s_p6_en
p2s_p8_en s2p_p9_cn s2p_p9_c GND
s2p_p10_c s2p_p10_on p2s_p10_c p2s_p10_cn p2s_p9_on s2p_p11_c VDD s2p_p11_o GND s2p_p10_e
p2s_p9_o s2p_p10_en VDD p2s_p10_on
p2s_p9_en p2s_p10_en s2p_p11_cn p2s_p11_en p2s_p11_e s2p_p11_on p2s_p11_c p2s_p11_cn p2s_p10_o s2p_p11_en s2p_p11_e p2s_p10_e UNUSED UNUSED UNUSED UNUSED UNUSED s2p_p13_on GND UNUSED VDD UNUSED VDD UNUSED GND UNUSED UNUSED lssd_scan_ in0 UNUSED lssd_scan_ out1 UNUSED UNUSED VDD UNUSED GND UNUSED GND UNUSED VDD UNUSED UNUSED lssd_scan_ out0 UNUSED lssd_scan_ in1 UNUSED UNUSED GND UNUSED VDD UNUSED VDD UNUSED GND UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED VDD UNUSED GND UNUSED GND UNUSED VDD UNUSED UNUSED UNUSED UNUSED UNUSED GND p2s_p11_o VDD UNUSED VDD p2s_p11_on UNUSED UNUSED UNUSED UNUSED
UNUSED s2p_p12_en UNUSED UNUSED GND s2p_p12_e
s2p_p14_o s2p_p13_o p2s_p12_c s2p_p12_o s2p_p12_on p2s_p12_e s2p_p12_cn s2p_p12_c s2p_p13_en p2s_p12_o p2s_p12_on p2s_p15_cn UNUSED p2s_p31_c VDD p2s_p13_c GND p2s_p12_cn VDD p2s_p12_en GND p2s_p13_on VDD s2p_p13_e
p2s_p15_c s2p_p15_on s2p_p14_on p2s_p13_cn p2s_p13_e s2p_p13_cn s2p_p13_c s2p_p14_e s2p_p14_en p2s_p13_o GND UNUSED VDD UNUSED GND UNUSED VDD p2s_p14_cn GND p2s_p13_en VDD p2s_p14_o GND p2s_p14_on
p2s_p30_cn p2s_p31_cn UNUSED p2s_p29_cn VDD UNUSED
s2p_p15_o p2s_p14_c p2s_p14_e s2p_p14_c p2s_p15_on s2p_p15_e s2p_p15_en UNUSED UNUSED UNUSED UNUSED s2p_p31_c VDD UNUSED GND UNUSED VDD s2p_p15_cn UNUSED UNUSED lssd_scan_ in3 UNUSED GND s2p_p14_cn VDD p2s_p15_o
s2p_p29_o s2p_p30_on s2p_p31_o p2s_p28_c GND p2s_p30_c
lssd_scan_ p2s_p15_e p2s_p14_en UNUSED in2 VDD UNUSED GND UNUSED VDD UNUSED p2s_p15_en lssd_scan_ out2 UNUSED lssd_scan_ out4 UNUSED lssd_scan_ in4 GND UNUSED VDD UNUSED GND UNUSED s2p_p15_c UNUSED UNUSED lssd_scan_ out3 UNUSED UNUSED
s2p_p29_c p2s_p29_en p2s_p30_en s2p_p31_cn s2p_p29_cn VDD p2s_p29_e GND
lssd_scan_ lssd_scan_ s2p_p30_en p2s_p30_on p2s_p31_o s2p_p30_cn p2s_p31_en out6 out5 s2p_p30_e GND s2p_p31_en VDD s2p_p30_c GND p2s_p31_e lssd_scan_ lssd_scan_ p2s_p30_e in6 in5
p2s_p29_on p2s_p30_o s2p_p31_e p2s_p31_on
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
281
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 41.
Scheduler Alpha Pin List
Signal Name
ce0_io ce0_scan ce0_testm3 GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND G16 J16 C07 B02 B06 B10 B14 B20 B24 B28 B32 D04 D08 D12 D16 D18 D22 D26 D30 F02 F06 F10 F14 F20 F24 F28 F32 H04 H08 H12 H16 H18 H22 H26 H30 K02 K06 K10 K14 K20 K24 K28 K32 M04 M08 M12 M16 M18 M22 M26 M30 P02 P06 P10 P14 P17 P20 P24 P28 P32 R15
Signal Name
GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND R19 T04 T08 T12 T16 T18 T22 T26 T30 U14 U17 U20 V04 V08 V12 V16 V18 V22 V26 V30 W15 W19 Y02 Y06 Y10 Y14 Y17 Y20 Y24 Y28 Y32 AB04 AB08 AB12 AB16 AB18 AB22 AB26 AB30 AD02 AD06 AD10 AD14 AD20 AD24 AD28 AD32 AF04 AF08 AF12 AF16 AF18 AF22 AF26 AF30 AH02 AH06 AH10 AH14 AH20 AH24
Signal Name
GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND GND jtag_tck jtag_tdi jtag_tdo jtag_tms jtag_trst_L lssd_ce1_a lssd_ce1_b lssd_ce1_c1 lssd_ce1_c2 lssd_ce1_c3 lssd_scan_in0 lssd_scan_in1 lssd_scan_in10 lssd_scan_in11 lssd_scan_in12 lssd_scan_in13 lssd_scan_in14 lssd_scan_in15 lssd_scan_in2 lssd_scan_in3 lssd_scan_in4 lssd_scan_in5 lssd_scan_in6 lssd_scan_in7 lssd_scan_in8 lssd_scan_in9 lssd_scan_out0 lssd_scan_out1 lssd_scan_out10 lssd_scan_out11 lssd_scan_out12 lssd_scan_out13 lssd_scan_out14 lssd_scan_out15 lssd_scan_out2 lssd_scan_out3 lssd_scan_out4 lssd_scan_out5 lssd_scan_out6 lssd_scan_out7 lssd_scan_out8 lssd_scan_out9 mon0 AH28 AH32 AK04 AK08 AK12 AK16 AK18 AK22 AK26 AK30 AM02 AM06 AM10 AM14 AM20 AM24 AM28 AM32 C03 C33 C31 C01 A07 A27 C29 A29 E29 A31 T25 V27 AL05 AL03 AL01 AJ03 V09 T07 AG31 AJ29 AN31 AN29 AN27 AE18 AG16 AL07 T27 V25 AN05 AN03 AJ05 AG03 V07 T09 AJ31 AL33 AL31 AL29 AL27 AG18 AE16 AN07 G07
282
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Signal Name
mon1 mon10 mon11 mon12 mon13 mon14 mon15 mon2 mon3 mon4 mon5 mon6 mon7 mon8 mon9 No Pin oob_ad0 oob_ad1 oob_ad2 oob_ad3 oob_ad4 oob_ad5 oob_ad6 oob_ad7 oob_clk oob_devsel0 oob_devsel1 oob_devsel2 oob_devsel3 oob_int_hi oob_int_lo oob_valid_L oob_wait_L p2s_p0_c p2s_p0_cn p2s_p0_e p2s_p0_en p2s_p0_o p2s_p0_on p2s_p10_c p2s_p10_cn p2s_p10_e p2s_p10_en p2s_p10_o p2s_p10_on p2s_p11_c p2s_p11_cn p2s_p11_e p2s_p11_en p2s_p11_o p2s_p11_on p2s_p12_c p2s_p12_cn p2s_p12_e p2s_p12_en p2s_p12_o p2s_p12_on p2s_p13_c p2s_p13_cn p2s_p13_e p2s_p13_en F07 N14 M13 M15 N15 P15 J13 E06 E07 A04 B05 H09 G08 L11 J09 A01 B01 C02 D03 E04 G06 H07 F05 G05 J08 U09 P13 R12 R14 K09 E02 N12 D01 E08 A06 G09 L12 A09 B09 L29 L30 P23 N24 N31 M33 N29 N30 N27 N26 R32 P33 AA25 AB27 AA28 AB29 AA32 AA33 AB25 AC27 AC28 AD29
Signal Name
p2s_p13_o p2s_p13_on p2s_p14_c p2s_p14_cn p2s_p14_e p2s_p14_en p2s_p14_o p2s_p14_on p2s_p15_c p2s_p15_cn p2s_p15_e p2s_p15_en p2s_p15_o p2s_p15_on p2s_p16_c p2s_p16_cn p2s_p16_e p2s_p16_en p2s_p16_o p2s_p16_on p2s_p17_c p2s_p17_cn p2s_p17_e p2s_p17_en p2s_p17_o p2s_p17_on p2s_p18_c p2s_p18_cn p2s_p18_e p2s_p18_en p2s_p18_o p2s_p18_on p2s_p19_c p2s_p19_cn p2s_p19_e p2s_p19_en p2s_p19_o p2s_p19_on p2s_p1_c p2s_p1_cn p2s_p1_e p2s_p1_en p2s_p1_o p2s_p1_on p2s_p20_c p2s_p20_cn p2s_p20_e p2s_p20_en p2s_p20_o p2s_p20_on p2s_p21_c p2s_p21_cn p2s_p21_e p2s_p21_en p2s_p21_o p2s_p21_on p2s_p22_c p2s_p22_cn p2s_p22_e p2s_p22_en p2s_p22_o AC33 AB31 AE28 AD27 AE29 AG32 AD31 AD33 AC24 AB23 AG30 AH31 AF33 AE31 F01 H05 J07 L09 J02 J01 J04 H03 N11 L08 L03 L02 L04 L05 N10 P11 M01 N03 N04 N05 N08 N07 P01 R02 C08 D09 L13 G10 B11 C11 AB07 AA09 AB05 AA06 AA01 AA02 AC07 AB09 AD05 AC06 AB03 AC01 AD07 AE06 AE05 AG02 AD01
Signal Name
p2s_p22_on p2s_p23_c p2s_p23_cn p2s_p23_e p2s_p23_en p2s_p23_o p2s_p23_on p2s_p24_c p2s_p24_cn p2s_p24_e p2s_p24_en p2s_p24_o p2s_p24_on p2s_p25_c p2s_p25_cn p2s_p25_e p2s_p25_en p2s_p25_o p2s_p25_on p2s_p26_c p2s_p26_cn p2s_p26_e p2s_p26_en p2s_p26_o p2s_p26_on p2s_p27_c p2s_p27_cn p2s_p27_e p2s_p27_en p2s_p27_o p2s_p27_on p2s_p28_c p2s_p28_cn p2s_p28_e p2s_p28_en p2s_p28_o p2s_p28_on p2s_p29_c p2s_p29_cn p2s_p29_e p2s_p29_en p2s_p29_o p2s_p29_on p2s_p2_c p2s_p2_cn p2s_p2_e p2s_p2_en p2s_p2_o p2s_p2_on p2s_p30_c p2s_p30_cn p2s_p30_e p2s_p30_en p2s_p30_o p2s_p30_on p2s_p31_c p2s_p31_cn p2s_p31_e p2s_p31_en p2s_p31_o p2s_p31_on AD03 AB11 AC10 AH03 AG04 AE03 AF01 AJ08 AK07 AC12 AG09 AM09 AN09 AL08 AM07 AG10 AC13 AL11 AM11 AK11 AJ11 AD13 AG11 AN12 AL13 AK13 AJ13 AE13 AG12 AN14 AM15 AH23 AC20 AJ22 AH21 AN21 AM21 AE22 AF23 AK25 AJ24 AL22 AN23 E11 D11 G11 K13 C13 A12 AH25 AE23 AN28 AJ25 AN24 AL24 AD23 AE24 AM29 AL28 AL25 AN26
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
283
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Signal Name
p2s_p3_c p2s_p3_cn p2s_p3_e p2s_p3_en p2s_p3_o p2s_p3_on p2s_p4_c p2s_p4_cn p2s_p4_e p2s_p4_en p2s_p4_o p2s_p4_on p2s_p5_c p2s_p5_cn p2s_p5_e p2s_p5_en p2s_p5_o p2s_p5_on p2s_p6_c p2s_p6_cn p2s_p6_e p2s_p6_en p2s_p6_o p2s_p6_on p2s_p7_c p2s_p7_cn p2s_p7_e p2s_p7_en p2s_p7_o p2s_p7_on p2s_p8_c p2s_p8_cn p2s_p8_e p2s_p8_en p2s_p8_o p2s_p8_on p2s_p9_c p2s_p9_cn p2s_p9_e p2s_p9_en p2s_p9_o p2s_p9_on plllock plltest_in plltest_out pwrup_reset_in_L ref_clk ref_clkn s2p_p0_c s2p_p0_cn s2p_p0_e s2p_p0_en s2p_p0_o s2p_p0_on s2p_p10_c s2p_p10_cn s2p_p10_e s2p_p10_en s2p_p10_o s2p_p10_on s2p_p11_c B15 A14 E14 C14 H13 G12 L20 F23 F21 E22 B21 A21 H23 J22 E24 D25 A23 C22 J23 F25 E25 A28 C24 A24 J24 K23 C28 B29 A26 C25 H29 F33 L25 J27 J33 J32 H31 J30 L26 N23 L32 L31 E05 G03 E03 A02 A05 C05 K11 J10 C09 A08 C06 D07 L27 M25 M31 L33 K29 L28 M27
Signal Name
s2p_p11_cn s2p_p11_e s2p_p11_en s2p_p11_o s2p_p11_on s2p_p12_c s2p_p12_cn s2p_p12_e s2p_p12_en s2p_p12_o s2p_p12_on s2p_p13_c s2p_p13_cn s2p_p13_e s2p_p13_en s2p_p13_o s2p_p13_on s2p_p14_c s2p_p14_cn s2p_p14_e s2p_p14_en s2p_p14_o s2p_p14_on s2p_p15_c s2p_p15_cn s2p_p15_e s2p_p15_en s2p_p15_o s2p_p15_on s2p_p16_c s2p_p16_cn s2p_p16_e s2p_p16_en s2p_p16_o s2p_p16_on s2p_p17_c s2p_p17_cn s2p_p17_e s2p_p17_en s2p_p17_o s2p_p17_on s2p_p18_c s2p_p18_cn s2p_p18_e s2p_p18_en s2p_p18_o s2p_p18_on s2p_p19_c s2p_p19_cn s2p_p19_e s2p_p19_en s2p_p19_o s2p_p19_on s2p_p1_c s2p_p1_cn s2p_p1_e s2p_p1_en s2p_p1_o s2p_p1_on s2p_p20_c s2p_p20_cn N25 N33 N32 M29 N28 AA30 AA29 Y33 W32 AA26 AA27 AC30 AC29 AB33 AA31 AA24 Y23 AE30 AF31 AC31 AC32 AA23 AC26 AH33 AF29 AE32 AE33 AE27 AC25 L10 M11 H01 J03 G04 F03 J06 K07 K03 K01 J05 G02 M09 L07 L01 M03 L06 K05 N09 M07 N02 N01 N06 M05 F09 J11 A10 C10 B07 E09 AA05 AA04
Signal Name
s2p_p20_e s2p_p20_en s2p_p20_o s2p_p20_on s2p_p21_c s2p_p21_cn s2p_p21_e s2p_p21_en s2p_p21_o s2p_p21_on s2p_p22_c s2p_p22_cn s2p_p22_e s2p_p22_en s2p_p22_o s2p_p22_on s2p_p23_c s2p_p23_cn s2p_p23_e s2p_p23_en s2p_p23_o s2p_p23_on s2p_p24_c s2p_p24_cn s2p_p24_e s2p_p24_en s2p_p24_o s2p_p24_on s2p_p25_c s2p_p25_cn s2p_p25_e s2p_p25_en s2p_p25_o s2p_p25_on s2p_p26_c s2p_p26_cn s2p_p26_e s2p_p26_en s2p_p26_o s2p_p26_on s2p_p27_c s2p_p27_cn s2p_p27_e s2p_p27_en s2p_p27_o s2p_p27_on s2p_p28_c s2p_p28_cn s2p_p28_e s2p_p28_en s2p_p28_o s2p_p28_on s2p_p29_c s2p_p29_cn s2p_p29_e s2p_p29_en s2p_p29_o s2p_p29_on s2p_p2_c s2p_p2_cn s2p_p2_e W02 Y01 AA07 AA08 AC05 AC04 AA03 AB01 Y11 AA10 AF03 AE04 AC02 AC03 AC08 AA11 AF05 AH01 AE01 AE02 AC09 AE07 AE10 AD11 AN08 AL09 AL06 AM05 AE11 AH09 AL10 AN10 AJ09 AN06 AF11 AE12 AN11 AL12 AJ10 AK09 AC14 AH11 AM13 AN13 AH13 AJ12 AJ21 AK21 AM19 AN20 AG22 AE21 AJ23 AK23 AL21 AN22 AG23 AD21 J12 H11 C12
284
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Signal Name
s2p_p2_en s2p_p2_o s2p_p2_on s2p_p30_c s2p_p30_cn s2p_p30_e s2p_p30_en s2p_p30_o s2p_p30_on s2p_p31_c s2p_p31_cn s2p_p31_e s2p_p31_en s2p_p31_o s2p_p31_on s2p_p3_c s2p_p3_cn s2p_p3_e s2p_p3_en s2p_p3_o s2p_p3_on s2p_p4_c s2p_p4_cn s2p_p4_e s2p_p4_en s2p_p4_o s2p_p4_on s2p_p5_c s2p_p5_cn s2p_p5_e s2p_p5_en s2p_p5_o s2p_p5_on s2p_p6_c s2p_p6_cn s2p_p6_e s2p_p6_en s2p_p6_o s2p_p6_on s2p_p7_c s2p_p7_cn s2p_p7_e s2p_p7_en s2p_p7_o s2p_p7_on s2p_p8_c s2p_p8_cn s2p_p8_e s2p_p8_en s2p_p8_o s2p_p8_on s2p_p9_c s2p_p9_cn s2p_p9_e s2p_p9_en s2p_p9_o s2p_p9_on soc_in soc_inn soc_out test_di1 A11 E10 F11 AM27 AL26 AM23 AL23 AC21 AG24 AK27 AJ26 AN25 AM25 AG25 AC22 E13 D13 A13 B13 E12 F13 D21 E21 A20 B19 J21 G22 D23 E23 A22 C21 K21 G23 C26 B27 C23 B23 G24 L21 E26 D27 B25 A25 L22 G25 M23 L24 J31 H33 F31 G30 K27 J28 K33 K31 J29 G32 C04 D05 B03 G18
Signal Name
test_di2 test_lt test_ri UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED A03 C27 J18 A15 A16 A17 A18 A19 A30 A32 A33 B17 B31 B33 C15 C16 C17 C18 C19 C20 C30 C32 D15 D17 D19 D29 D31 D33 E15 E16 E17 E18 E19 E20 E27 E28 E30 E31 E32 E33 F15 F17 F19 F27 F29 G01 G13 G14 G15 G17 G19 G20 G21 G26 G27 G28 G29 G31 G33 H15 H17
Signal Name
UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED H19 H21 H25 H27 J14 J15 J17 J19 J20 J25 J26 K15 K17 K19 K25 L14 L15 L16 L17 L18 L19 L23 M17 M19 M21 N16 N17 N18 N19 N20 N22 P03 P05 P07 P09 P19 P21 P25 P27 P29 P31 R01 R03 R04 R05 R06 R07 R08 R09 R10 R11 R13 R16 R17 R18 R20 R21 R22 R23 R24 R25
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
285
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Signal Name
UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED R26 R27 R28 R29 R30 R31 R33 T01 T03 T05 T11 T13 T15 T19 T21 T23 T29 T31 T33 U01 U02 U03 U04 U05 U06 U07 U08 U10 U11 U12 U13 U15 U19 U21 U22 U23 U24 U25 U26 U27 U28 U29 U30 U31 U32 U33 V01 V03 V05 V11 V13 V15 V19 V21 V23 V29 V31 V33 W01 W03 W04
Signal Name
UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED W05 W06 W07 W08 W09 W10 W11 W12 W13 W14 W16 W17 W18 W20 W21 W22 W23 W24 W25 W26 W27 W28 W29 W30 W31 W33 Y03 Y05 Y07 Y09 Y13 Y15 Y19 Y21 Y25 Y27 Y29 Y31 AA12 AA14 AA15 AA16 AA17 AA18 AA19 AA20 AA22 AB13 AB15 AB17 AB19 AB21 AC11 AC15 AC16 AC17 AC18 AC19 AC23 AD09 AD15
Signal Name
UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED AD17 AD19 AD25 AE08 AE09 AE14 AE15 AE17 AE19 AE20 AE25 AE26 AF07 AF09 AF13 AF15 AF17 AF19 AF21 AF25 AF27 AG01 AG05 AG06 AG07 AG08 AG13 AG14 AG15 AG17 AG19 AG20 AG21 AG26 AG27 AG28 AG29 AG33 AH05 AH07 AH15 AH17 AH19 AH27 AH29 AJ01 AJ02 AJ04 AJ06 AJ07 AJ14 AJ15 AJ16 AJ17 AJ18 AJ19 AJ20 AJ27 AJ28 AJ30 AJ32
286
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Signal Name
UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED UNUSED VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD AJ33 AK01 AK03 AK05 AK15 AK17 AK19 AK29 AK31 AK33 AL02 AL04 AL14 AL15 AL16 AL17 AL18 AL19 AL20 AL30 AL32 AM01 AM03 AM17 AM31 AM33 AN01 AN02 AN04 AN15 AN16 AN17 AN18 AN19 AN30 AN32 AN33 B04 B12 B18 B26 D10 D20 D28 D32 F04 F08 F16 F22 H02 H14 H24 H28 K08 K12 K18 K30 M06 M24 M32 N13
Signal Name
VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD N21 P04 P16 P18 P26 T02 T10 T14 T17 T20 T28 U16 U18 V06 V14 V17 V20 V24 V32 Y08 Y16 Y18 Y30 AA13 AA21 AB02 AB10 AB28 AD04 AD16 AD22 AD26 AF06 AF10 AF20 AF32 AH12 AH18 AH26 AH30 AK02 AK06 AK14 AK24 AM08 AM16 AM22 AM30 F30 H32 K26 M28 P22 P30 T24 T32 V28 Y22 Y26 AB24 AB32
Signal Name
VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDD VDDA AD30 AF28 AK32 AB14 AB20 AD12 AD18 AF14 AF24 AH08 AH16 AH22 AK10 AK20 AK28 AM04 AM12 AM18 AM26 D02 H06 K04 M02 M10 P08 P12 T06 V02 V10 Y04 Y12 AB06 AD08 AF02 AH04 B08 B16 B22 B30 D06 D14 D24 F12 F18 F26 H10 H20 K16 K22 M14 M20 E01
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
287
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
5.6.2
Package Dimensions
This section outlines the mechanical dimensions for the Scheduler device. The package is a 1088 ceramic column grid array (CCGA). NOTE: Drawings are not to scale.
Figure 67. Scheduler CCGA package Dimensions - Top and Side Views
D1
E1
E
Top View A Pin A01 corner A1
A2
aaa
288
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 68. Scheduler CCGA package Dimensions - Bottom View
b
e
AN AM AL AK AJ AH AG AF AE AD AC AB AA Y W V U T R P N M L K J H G F E D C B A 02 04 03 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 01
Pin A01 corner
Bottom View
Table 42. Scheduler CCGA Mechanical Specifications
e = 1.27 mm pitch Symbol Minimum A A1 A2 aaa D D1 E E1 M N b 0.48 NOTE: Column alloy is 90/10 PbSn Nominal 6.83 4.62 2.21 0.15 42.50 40.64 42.50 40.64 33 x 33 1088 0.52 mm Maximum mm mm mm mm mm mm mm mm Units
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
289
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
5.6.3
Part Number
The PMC-Sierra Part Number for the Scheduler is: Scheduler PM9311-UC
PMC Logo
45 L9 37 4 P Q PM9311-UC Scheduler Re v A BXXLL LLLLLL
Country of Origin
Source "B" Part Number PMC Part Number Device Name Revision Lot Number Date Code
BYYW W X XX X XX X XX X XX
Terminal A01 Identifier
PM 9 3 11 - U C
PMC prefix IRD prefix IC Chipset family code, 1=TT1, 5 =TTX Sequential part number (1=SCH, 2=XBAR, 3=DS, 5=EPP) Temp. range (C = Commercial 0-70 degrees C) Package code (U = CCGA, H = CBGA)
290
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
291
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
292
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
6
6.1
Characteristics
SIGNAL ASSOCIATIONS
Table 43. Signal Associations
Symbol
Device
Associated Signals
3.3V-tolerant 2.5V CMOS (OOB and Test Interfaces)
DS Vol1, Voh1 Ioh1, Iol1, Zout1, tval1, tsus1, tr1, tf1 EPP XBAR SCHED
mon[15:0], oob_ad[7:0], oob_int_hi[1:0], oob_int_lo[1:0], oob_wait_L, plllock, lssd_scan_out[15:0], plltest_out oob_ad[7:0], oob_int_hi, oob_int_lo, oob_wait_L, soc_out, lssd_scan_out[15:0], plltest_out M_ackreg_L, M_pending_L, M_xreset_L, oob_ad[7:0], oob_int_hi, oob_int_lo, soc_out, plllock, lssd_scan_out[15:0], plltest_out mon[15:0], oob_ad[7:0], oob_int_hi, oob_int_lo, soc_out, plllock, lssd_scan_out[15:0], plltest_out oob_ad[7:0], oob_clk, oob_devsel0, oob_devsel1, oob_devsel2, oob_valid_L, pwrup_reset_L, crdcrcen0, crdcrcen1, crden0, crden1, ibpen0, ibpen1, ce0_io, ce0_scan, ce0_tstm3, lssd_ce1_a, lssd_ce1_b, lssd_ce1_c1, lssd_ce1_c2,, lssd_ce1_c3, lssd_scan_in[15:0], plltest_in, test_di1, test_di2, test_lt, test_re, test_ri oob_ad[7:0], oob_clk, oob_valid_L, pwrup_reset_in_L, ce0_io, ce0_scan, ce0_test, ce0_tstm3, lssd_ce1_a, lssd_ce1, plltest_in_b, lssd_ce1_c1, lssd_ce1_c2, lssd_ce1_c3, lssd_scan_in[15:0], test_di1, test_di2, test_lt, test_re, test_ri oob_ad[7:0], oob_clk, oob_valid_L, oob_devsel0, oob_devsel1, oob_devsel2, oob_devsel3, oob_valid_L, pwruprst_L, ce0_io, ce0_scan, lssd_ ce1_a, lssd_ce1_b, lssd_ce1_c1, lssd_ce1_c2, lssd_scan_in[15:0], plltest_in, test_di1, test_di2, test_lt, test_ri oob_ad[7:0], oob_clk, oob_valid_L, oob_devsel0, oob_devsel1, oob_devsel2, oob_devsel3, oob_valid_L, pwrup_reset_in_L, ce0_io, ce0_scan, ce0_testm3, lssd_ce1_a, lssd_ce1_b. lssd_ce1_c1, lssd_ce1_c2, lssd_ce1_c3, lssd_scan_in[15:0], plltest_in, test_di1, test_di2, test_lt, test_ri oob_clk oob_clk oob_clk oob_clk
DS
EPP Vih1, Vil1, tsu1, th1 XBAR
SCHED
DS tper1, tph1, tpl1 EPP XBAR SCHED
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
293
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 43. Signal Associations (Continued)
Symbol
Device
Associated Signals
2.5V CMOS (JTAG Interface)
Vol2, Voh2,Ioh2, Iol2, Zout2, tval2, tsus2, tr2, tf2
DS EPP XBAR SCHED DS
jtag_tdo jtag_tdo jtag_tdo jtag_tdo jtag_tdi, jtag_tms, jtag_trst_L jtag_tdi, jtag_tms, jtag_trst_L jtag_tdi, jtag_tms, jtag_trst_L jtag_tdi, jtag_tms, jtag_trst_L jtag_tck jtag_tck jtag_tck jtag_tck 3.3V-tolerant 2.5V CMOS (Serdes Interface)
Vih2, Vil2, tsu2, th2
EPP XBAR SCHED DS
tper2, tph2, tpl2
EPP XBAR SCHED
Vol3, Voh3, Ioh3, Iol3, Zout3, tval3, tsus3, tr3, tf3 Vih3, Vil3, tsu3, th3
DS
fotx0_txd[9:0], pin_diode0_reset_L, vcsel0_reset_L, xcvr0_loopback, fotx1_ txd[9:0], pin_diode1_reset_L,, vcsel1_reset_L, xcvr1_loopback, tx_clkin, tx_clkout, fotx_clk_out forx0_clk, forx0_rxd[9:0], pin_diode0_sig_det, vcsel0_laser_act, xcvr0_com_det, xcvr0_sig_det, forx1_clk, forx1_rxd[9:0], pin_diode1_sig_det, vcsel1_laser_act, xcvr1_com_det, xcvr1_sig_det, fotx_clk_in forx0_clk, forx1_clk, fotx_clk_in forx0_clk, forx1_clk, fotx_clk_in forx0_clk, forx1_clk, fotx_clk_in forx0_clk, forx1_clk, fotx_clk_in
DS DS
tper3, tph3, tpl3
EPP XBAR SCHED
294
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 43. Signal Associations (Continued)
Symbol
Device
Associated Signals
200 Mbps HSTL (EPP/DS Interface)
Voh4, Vol4, tval4, tsus4, tr4, tf4
DS (HSTL Class 1) EPP (HSTL Class 1) DS EPP
d2p_d0_id[7:0], d2p_d0_od[7:0], d2p_d1_id[7:0], d2p_d1_od[7:0]
p2d_d0_id[7:0], p2d_d1_id[7:0], p2d_d2_id[7:0], p2d_d[3:0]_ic[7:0], p2d_d[3:0]_oc[7:0] p2d_ic[7:0], p2d_oc[7:0], p2d_d0_id[7:0], p2d_d1_id[7:0] d2p_d0_id[7:0], d2p_d0_od[7:0], d2p_d1_id[7:0], d2p_d1_od[7:0], d2p_d2_id[7:0], d2p_d2_od[7:0] 800 Mbps STI (AIB Interfaces)
Vih4, Vil4, Zterm4, tsu4, th4
DS Vocomm5, Vodiff5, Zout5, tval5, tsus5, tr5, th5 EPP XBAR SCHED DS Vicomm5, Vidiff5 EPP XBAR SCHED DS tper5, tph5, tpl5 EPP XBAR SCHED
d2x_d0a_[c, cn, e, en, o, on], d2x_d0b_[c, cn, e, en, o, on], d2x_d1a_[c, cn, e, en, o, on], d2x_d1b_[c, cn, e, en, o, on] p2s_s0_[c cn, e, en, o, on], p2s_s1_[c cn, e, en, o, on] x2d_p[31:0]_[c, cn, e, en, o, on] s2p_p[31:0]_[c, cn, e, en, o, on] x2d_d0a_[c, cn, e, en, o, on], x2d_d0b_[c, cn, e, en, o, on], x2d_d1a_[c, cn, e, en, o, on], x2d_d1b_[c, cn, e, en, o, on] s2p_s0_[c cn, e, en, o, on], s2p_s1_[c cn, e, en, o, on] d2x_p[31:0]_[c, cn, e, en, o, on] s2p_p[31:0]_[c, cn, e, en, o, on] x2d_d0a_[c,cn], x2d_d0b_[c,cn], x2d_d1a_[c,cn], x2d_d1b_[c,cn] s2p_s0_[c,cn], s2p_s1_[c,cn] d2x_p[31:0]_[c,cn] s2p_p[31:0]_[c,cn]
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
295
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 43. Signal Associations (Continued)
Symbol
Device
PECL DS
Associated Signals
ref_clk, ref_clkn, soc_in, soc_inn ref_clk, ref_clkn, soc_in, soc_inn ref_clk, ref_clkn, soc_in, soc_inn ref_clk, ref_clkn, soc_in, soc_inn ref_clk, ref_clkn ref_clk, ref_clkn ref_clk, ref_clkn ref_clk, ref_clkn
Vicomm6, Vidiff6
EPP XBAR SCHED DS
tper6, tph6, tpl6
EPP XBAR SCHED
296
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
6.2
ABSOLUTE MAXIMUM RATINGS
Table 44.
Absolute Maximum Ratings
Symbol
VDD Tship Tstg Tlead
Parameter
Supply Voltage Shipping StorageTemp Lead Temp ESDProtection
Conditions
Min
2.4
Typ
Max
2.6 60 30
Units
V
Dry Pack 60%RH Soldering, 60s (IR or Vapor) (Phase Reflow)
-40 0
oC
220 1 kV
Table 45.
Absolute Maximum Ratings for 3.3V-tolerant 2.5V CMOS (OOB & Serdes Interface)
Symbol
DCin DCout
Parameter
Input Voltage
Conditions
Min
-0.6
Typ
Max
3.9
Units
V
Output Voltage
Table 46. Absolute Maximum Ratings for 2.5V CMOS (JTAG Interface)
Symbol
DCin DCout
Parameter
Input Voltage
Conditions
Min
-0.6
Typ
Max
VDD+0.6
Units
V
OutputVoltage
Table 47.
Absolute Maximum Ratings for 200 Mbps HSTL (EPP/DS Interface)
Symbol
DCin DCout VDDQin
Parameter
Input Voltage
Conditions
Min
-0.6
Typ
Max
VDD+0.6
Units
OutputVoltage HSTL Output Supply V 1.5 1.6 1.7
V
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
297
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 48. Absolute Maximum Ratings for 800 Mbps STI (AIB Interface)
Symbol
DCin DCout
Parameter
Input Voltage
Conditions
Min
-0.6
Typ
Max
VDD+0.6
Units
V
OutputVoltage
Table 49.
Absolute Maximum Ratings for 2.5V 200 Mbps PECL (CLK and SOC Signals)
Symbol
DCin DCout
Parameter
Input Voltage
Conditions
Min
-0.6
Typ
Max
VDD+0.6
Units
V
OutputVoltage
298
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
6.3
RECOMMENDED OPERATING CONDITIONS
Table 50.
Recommended Operating Conditions
Symbol
VDD VDDQ Fhssch, Fhsxbar
Parameter
Supply Voltage HSTL Output Supply Voltage Mechanically Isolated Heat Sink Applied Force Mechanically Isolated Heat Sink Applied Force Attached Heat Sink Mass Attached Heat Sink Mass JuncOperTemp EPP Heat Transfer
Conditions
Min
2.4 1.5
Typ
Max
2.6
Units
V
1.6
1.7
23
lbs
Fhsepp
13
lbs
Mhsds, Mhsepp
30 50 10 85 0.36
g g
oC
Mhsxbar, Mhssch
Tj jcepp jcds jcxbar jcsch PDepp6 PDepp7
DS Heat Transfer XBAR Heat Transfer SCH Heat Transfer EPP Power Diss EPP Power Diss
Incident air velocity = 2 m/s
o
0.36 0.26
C/W
6 DS Loads ref_clk =200mHz 7 DS Loads ref_clk =200mHz EPP Header Data Connections ref_clk =200mHz No Header Connections ref_clk =200mHz ref_clk =200mHz
8.68 8.68
9.62 9.62
10.60 10.60
PDds01
DS Power Diss
3.82
4.24
4.67 W
PDds45 PDxbar PDsch
DS Power Diss XBAR Power Diss SCH Power Diss
3.65 9.52 19.04
4.04 10.55 21.10
4.45 11.63 23.26
NOTE: Power dissipation is based on 2.5 V +/-5% supply. Power is dissipated within the package, and can be used for heat transfer calculations. Power dissipation for 1.5V HSTL output is primarily external to the device package, through the termination resistors, see Table 51.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
299
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
300
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 51.
Additional Power Design Requirements
Symbol
PDhstl1
Parameter
Termination Power per HSTL Class 1 output
Conditions
ref_clk =200mHz
Min
Typ
22.2a
Max
Units
mW
a. These power figures are not included as part of the heat calculation numbers. The 1.6V supply current would be based on the HSTL termination resistance scheme used in Figure 80 of the Characteristics section. This is dependent on the termination scheme used in the system and the tolerance of the resistors used for termination.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
301
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
6.4
DC ELECTRICAL CHARACTERISTICS
Table 52.
DC Electrical Characteristics for 3.3V-tolerant 2.5V CMOS (OOB Interface)
Symbol
Voh1 Vol1 Ioh1
Parameter
Output High V Output Low V Output High I
Conditions
Min
2.0 0
Typ
Max
VDD
Units
V
0.4
Voh@2.0V, VDD@2.3V Tj=85oC Vol@0.4V VDD@2.3V Tj=85oC
7a mA
Iol1 Zout1 Vih1 Vil1
Output Low I OutputImpedance Input High V InputLowV
9a 50 1.7 V 0 0.7 Ohm
a. This parameter is not tested. It is provided here as a reasonable guide to design, based on expected process parameters.
Table 53. DC Electrical Characteristics for 2.5V CMOS (JTAG Interface)
Symbol
Voh2 Vol2 Ioh2
Parameter
Output High V Output Low V Output High I
Conditions
Min
2.0 0
Typ
Max
VDD
Units
V
0.4
Voh@2.0V, VDD@2.3V Tj=85oC Vol@0.4V VDD@2.3V Tj=85 oC
7a mA
Iol2 Zout2 Vih2 Vil2
Output Low I OutputImpedance Input High V InputLowV
9a 50 1.7 0 VDD V 0.7 Ohm
a. This parameter is not tested. It is provided here as a reasonable guide to design, based on expected process parameters.
302
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 54.
DC Electrical Characteristics for 3.3V-tolerant 2.5V CMOS (Serdes)
Symbol
Voh3 Vol3 Ioh3
Parameter
Output High V Output Low V Output High I
Conditions
Min
1.9 0
Typ
Max
VDD
Units
V
0.4
Voh@2.0V, VDD@2.3V Tj=85oC Vol@0.4V VDD@2.3V Tj=85oC
7a mA
Iol3 Zout3 Vih3 Vil3
Output Low I OutputImpedance Input High V InputLowV
9a 50 1.7 V 0 0.7 Ohm
a. This parameter is not tested. It is provided here as a reasonable guide to design, based on expected process parameters.
Table 55. DC Electrical Characteristics for 200 Mbps HSTL (EPP/DS Interface)
Symbol
VDDQa Vref
Parameter
HSTLSupply V HSTL Ref V Output High V (Class 2)
Conditions
Min
1.5 0.7
Typ
1.6 0.75
Max
1.7 0.8
Units
Ioh=16mA@1.0V VDDQ -0.4 Ioh=8mA@1.0V Iol=16mA@0.4V 0.4 Iol=8mA@0.4V VREF +0.05 -0.5 R1=R2=100 Ohm Req=50 Ohm 100 VDDQ +.0v VREF -0.05 Ohm V
Voh4 Output High V (Class 1) Output Low V (Class 2) Vol4 Output Low V (Class 1) Vih4 Vil4 Zterm4 (HSTL Class 1) Input High V InputLowV Load Term
a. VDDQ is recommended to be 1.6 V to provide additional noise margin above 0.75 V for HSTL signals.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
303
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 56.
DC Electrical Characteristics for 800 Mbps STI (AIB Interfaces)
Symbol
Vocomm5 Vodiff5 Zout5 Vicomm5 Vidiff5 Zterm5
Parameter
OutputCommMode OutputDiffSwing OutputImpedance InputCommMode Input Diff Swing Term Impedance
Conditions
Min
1.05 0.8
Typ
Max
1.45
Units
V
1.2 20 1.25 V Ohm
0.4 100 ohm Line-to-Line in Receiver 100 Ohm
Table 57. DC Electrical Characteristics for 2.5V 200 Mbps PECL (CLK and SOC Signals)
Symbol
Vicomm6 Vidiff6
Parameter
Input Comm Mode Input Diff Swing
Conditions
Min
Typ
1.3
Max
Units
V
0.4
304
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
6.5
AC ELECTRICAL CHARACTERISTICS
Table 58.
AC Electrical Characteristics for 3.3V-tolerant 2.5V CMOS (OOB Interface)
Symbol
tsu1 th1 tval1 tsus1 tr1, tf1 tper1 tph1 tpl1
Parameter
Input Setup Input Hold OutputValid OutputSustain Output Rise/Fall Time Clock Period ClockPulseHigh ClockPulseLow
Conditions
Trf=2ns Referenced to OOB_clock. R1=50 Ohm, C1=1 pF Referenced to OOB_clock.
Min
7.6 2.4
Typ
Max
Units
12a 1.7 ns 1.0a 40 Defines OOB_clock. 20
a. This parameter is not tested. It is provided here as a reasonable guide to design, based on expected process parameters.
Table 59. AC Electrical Characteristics for 2.5V CMOS (JTAG Interface)
Symbol
tsu2 th2 tval2 tsus2 tr2, tf2 tper2 tph2 tpl2
Parameter
Input Setup Input Hold OutputValid OutputSustain Output Rise/Fall Time Clock Period ClockPulseHigh ClockPulseLow
Conditions
Trf=2ns Referenced to jtag_clock.
Min
24a 13a
Typ
Max
Units
37a R1=50 Ohm, C1=1 pF Referenced to jtag_clock. 20a 1.0 100 Defines jtag_clock. 45a
a
ns
a. This parameter is not tested. It is provided here as a reasonable guide to design, based on expected process parameters.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
305
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 60. AC Electrical Characteristics for 3.3V-tolerant 2.5V CMOS (Serdes Interface)
Symbol
tsu3 th3 tval3 tsus3 tr3, tf3 tper3 tph3 tpl3
Parameter
Input Setup Input Hold OutputValid OutputSustain Output Rise/Fall Time Clock Period ClockPulseHigh ClockPulseLow
Conditions
Trf=500 ps Referenced to forx0_clk, forx1_clk
Min
453
Typ
Max
Units
ps
970 2.45a
R1=50 Ohm, C1=1 pF Referenced to fotx_clk_out, Defines fotx_clk_in, forx0_clk, forx1_clk
2.87 0.7a 6.67 3.33 ns
a. This parameter is not tested. It is provided here as a reasonable guide to design, based on expected process parameters.
Table 61.
Reference Clock for 200 Mbps HSTL (EPP/DS Interface)
Symbol
tsu4 th4 tval4 tsus4 tr4, tf4
Parameter
Input Setup Input Hold OutputValid OutputSustain Output Rise/Fall Time
Conditions
Trf=700 ps Referenced to ref_clk, ref_clkn Class1: R1=R2=100 Ohm, C1=1 pF Referenced to ref_clk, ref_clkn
Min
530 500
Typ
Max
Units
2200 800 520a
ps
a. This parameter is not tested. It is provided here as a reasonable guide to design, based on expected process parameters. NOTE: Does not include Static Phase Offset or Jitter ( See Table 64 on page 308)
306
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 62. AC Electrical Characteristics for 800 Mbps STI (AIB Interface)
Symbol
tsu5
Parameter
Input Setup
Conditions
Trf=750 ps Referenced to s2p_s[1:0]_[c,cn] x2d_d[1:0][a,b]_[c,cn] d2x_p[31:0]_[c,cn] s2p_p[31:0]_[c,cn] R1=100 Ohm, C1=C2=1 pF Referenced to s2p_s[1:0]_[c,cn] x2d_d[1:0][a,b]_[c,cn] d2x_p[31:0]_[c,cn] s2p_p[31:0]_[c,cn] Swing = 0.8V Vlo = 0.761V Vhi = 1.59V R1=100 Ohm, C1=C2=1 pF Referenced to s2p_s[1:0]_[c,cn] x2d_d[1:0][a,b]_[c,cn] d2x_p[31:0]_[c,cn] s2p_p[31:0]_[c,cn] Defines s2p_s[1:0]_[c,cn] x2d_d[1:0][a,b]_[c,cn] d2x_p[31:0]_[c,cn] s2p_p[31:0]_[c,cn]
Min
190
Typ
Max
Units
th5
Input Hold
180
tval5
OutputValid
382a
tsus5
OutputSustain
432
ps
tr5
Output Rise Time
645
tf5
Output Fall Time
386
tper5 tph5 tpl5
Clock Period ClockPulseHigh ClockPulseLow
2.5 ns 1.10 1.38
a. This parameter is not tested. It is provided here as a reasonable guide to design, based on expected process parameters. NOTE: Does not include Static Phase Offset or Jitter ( See Table 64 on page 308)
Table 63. AC Electrical Characteristics for 2.5V 200Mbps PECL (CLK Signals)
Symbol
tsu6 th6 tper6 tph6 tpl6
Parameter
Input Setup Input Hold Clock Period ClockPulseHigh ClockPulseLow
Conditions
Trf=600 ps Referenced to ref_clk, ref_clkn Defines ref_clk, ref_clkn
Min
1.3 0.3a
Typ
Max
Units
5 2.5
ns
a.. This parameter is not tested. It is provided here as a reasonable guide to design, based on expected process parameters
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
307
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 64.
Jitter and Static Phase Offset for PLL
Symbol
tccj
Parameter PLL Jitter - Cycle to Cycle PLL Long-term Jitter or Device to Device PLL Static Phase Offset
Conditions
Min
Typ
Max
100a
Units
tltj
400a
ps
tpos
200a
a. This parameter is not tested. It is provided here as a reasonable guide to design, based on expected process parameters.
6.6
TIMING DIAGRAMS
Figure 69. Setup and Hold for 3.3V-tolerant 2.5V CMOS and 2.5V CMOS (OOB Interface)
Reference Clock
tval1 Output tsus1
Input
tsu1
th1
308
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 70. Rise and Fall Times for 3.3V-tolerant 2.5V CMOS and 2.5V CMOS (OOB Interface)
tper1
50% tph1 tpl1
80%
20%
tr1
tf1
Figure 71. OOB Test Loading
R1=50 Ohm Output C1=1pF Test Point
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
309
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 72. Setup and Hold for 3.3V-tolerant 2.5V CMOS and 2.5V CMOS (JTAG Interface)
Reference Clock
tval2 Output tsus2
Input
tsu2
th2
Figure 73. Rise and Fall Times for 3.3V-tolerant 2.5V CMOS and 2.5V CMOS (JTAG Interface)
tper2
50% tph2 tpl2
80%
20%
tr2
tf2
Figure 74. JTAG Test Loading
R1=50 Ohm Output Test Point
C1=1pF
310
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 75. Setup and Hold for 3.3V-tolerant 2.5V CMOS and 2.5V CMOS (Serdes Interface)
Reference Clock
tval3 Output tsus3
Input
tsu3
th3
Figure 76. Rise and Fall Times for 3.3V-tolerant 2.5V CMOS and 2.5V CMOS (Serdes Interface)
tper3
50% tph3 tpl3
80%
20%
tr3
tf3
Figure 77. Serdes Test Loading
R1=50 Ohm Output C1=1pF Test Point
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
311
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 78. Setup and Hold for 200 Mbps HSTL (EPP/DS Interface)
Reference Clock
tval4 Output tsus4
Input
tsu4
th4
Figure 79. Rise and Fall Times for 200 Mbps HSTL (EPP/DS Interface)
80%
20%
tr4
tf4
312
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 80. Class 1 HSTL Test Loading
Class 1 Driver VDDQ
R1=100 Ohm Test Point Output C1=1pF R2=100 Ohm
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
313
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 81. Setup and Hold for 800 Mbps STI (AIB Interfaces)
Reference Clock
Output
tval5
tsus5
tval5
tsus5
Input
tsu5
th5
tsu5
th5
Figure 82. Rise and Fall Times for 800 Mbps STI (AIB Interfaces))
tper5
50% tph5 tpl5
80%
20%
tr5
tf5
80% Differential Voltage Swing Differential Crossover Range 50% 20% Commom Mode Voltage Level
314
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 83. AIB Test Loading
Test Point True Signal C1=1pF
Output
R1=100 Ohm C2=1pF Compliment Signal Test Point
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
315
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 84. Setup and Hold for 2.5V 200 Mbps PECL (CLK & SOC Signals)
Reference Clock
Input
tsu6
th6
Figure 85. Clock Period and Duty Cycle for 2.5V 200 Mbps PECL (CLK Signal)
tper6
50% tph6 tpl6
316
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Appendix A. General Packet/Frame Formats
A.1 FRAME FORMATS - DATASLICE TO AND FROM ENHANCED PORT PROCESSOR
Table 65 and Table 66 show the format of the Request/Grant from iDS and Data from oDS frames, Table 67 shows the format of the Grant/Data to iDS frames, and Table 68 shows the format of the Control to xDS frames.
Table 65. Request/Data from iDS Frame Format
Field
Reserved VLD Code Error Token Request Reserved Cell Data Reserved
Size (bits)
1 1 1 1 4 48 8 Table 66.
Meaning
1 if frame contains valid cell 1 if frame contained 8b/10b error when received 1 if DS requests token to send data to EPP
Cell Data (includes LCS header)
Data from oDS Frame Format
Field
Reserved VLD Reserved Cell Data Reserved
Size (bits)
1 1 6 48 8 Table 67.
Meaning
1 if frame is valid (used on oDS/oEPP only)
Cell Data (includes LCS header)
Grant/Data to iDS Frame Format
Field
Reserved
Size (bits)
8
Meaning
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
317
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 67.
Grant/Data to iDS Frame Format
Field
Cell Data Credit CRC or Reserved
Size (bits)
48 8
Meaning
Cell Data (includes LCS header with CRC over Tag only) CRC over the credit portion of the LCS header on DS1, reserved on DS0, DS2 --DS11
Table 68.
Control to xDS Link Format
Field
Reserved Read Zero/ Token Rtag VLD RTag Read Req Read Addr. Write Req. Write Addr. Reserved
Size (bits)
1 1 1 5 1 23 1 23 8
Meaning
Output Side: 1 if DS is to send all-0 cell into oFIFO or Crossbar Input Side: 1 if DS now has a token to use for sending cells into the EPP 1 if the routing tag is valid Routing Tag (only used by iEPP) 1 if read is to be performed from iQM or oQM Read Address of cell to be read from iQM or oQM 1 if write is to be performed to iQM or oQM Write Address of cell to be written into iQM or oQM
A.2
FRAME FORMATS - DATASLICE TO AND FROM CROSSBAR
Table 69 shows the frame format from the Dataslices to the Crossbar; Table 70 shows the frame format used from the Crossbar to the Dataslices.
Table 69. Dataslice to Crossbar Frame Format
Field
synched Cell VLD RTagVLD RTag Cell Data CRC
Size (bits)
1 1 1 5 48 8
Meaning
AIB synched 1 if the cell is valid Routing Tag Valid Routing Tag Cell Data CRC computed over previous 56 bits
318
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 70.
Crossbar to Dataslice Frame Format
Field
synched VLD Reserved Cell Data CRC
Size (bits)
1 1 6 48 8
Meaning
AIB synched 1 if cell data is valid
Cell Data CRC computed over previous 56 bits
A.3 FRAME FORMATS - ENHANCED PORT PROCESSOR TO AND FROM SCHEDULER
Table 71 shows the frame format from the Scheduler to the PP. Table 72 shows the frame format from the EPP to the Scheduler. When the "Request Valid" is set to 0 but the "M/U" bit is set to 1, the frame is a special control frame for the Scheduler.
Table 71. Scheduler to EPP Frame Format
Field
synched Grant VLD Grant DQ Reserved TDM Grant Grant M/U Grant Priority Grant Port Reserved RTag VLD RTag Port Freeze Flow Control Crossbar Sync
Size (bits)
1 1 1 4 1 1 2 5 2 1 5 1
Meaning
AIB Synched 1 if the grant is valid 1 if the grant should cause the cell to be dequeued
1 if the grant is for a TDM cell 1 if the grant is multicast, 0 if unicast The priority of the grant The output port of the grant
1 if the routing tag is valid The input port of the routing tag 1 if the EPP should be frozen, 0 if not.
1
Flow Control Crossbar Sync bit
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
319
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 71.
Scheduler to EPP Frame Format (Continued)
Field
Reserved Sync TDM Table Sel Reserved CRC
Size (bits)
6 1 1 22 8
Meaning
if set, PP should set outgoing LCS TDM Sync bit if 0, use TDM table 1, if 1, use TDM table 2.
CRC computed over previous 56 bits
320
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Table 72.
EPP to Scheduler Frame Format
Field
synched BP VLD UBP VLD UBP Input TDM Req. Output TDM Req. TDM Port Suggested TDM Sync TDM Table Select Request VLD M/U and control msg. Priority Fanout CRC
Size (bits)
1 1 1 8 1 1 5 1 1 1
Meaning
AIB Synched 1 if the QID being read off the BP FIFO should be backpressured 1 if the unbackpressure information is valid QID (cast, priority, input port [if unicast, input port ignored if multicast]) for which backpressure is deasserted. Asserted when input has TDM cell to send. Asserted when output is ready to receive TDM cell Input port from which the output is ready to receive TDM cell (valid only if Output TDM asserted) 1 if this port is currently sending a suggested TDM Sync. TDM Frame to use next time the Sync is set (only used if this port happens to be chosen as Master by the Scheduler) 1 if the best-effort request to follow is valid. If Request VLD is set then this bit indicates if the request is multicast (set to 1) or unicast (set to 0). If Request VLD is not set then this bit indicates if this frame is a control message to the Scheduler (set to 1) or just a frame with no request (set to 0). Priority of request Multicast fanout if request is multicast, top 5 bits used for output port and bottom 27 bits reserved if the request is unicast. CRC computed over previous 58 bits
1 2 32 8
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
321
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
322
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Appendix B. Common Pinout Configuration
B.1 JTAG INTERFACE
The ETT1 Chip Set supports the following JTAG public instructions in conformance with the IEEE Std 1149.1 specification: * * * * IDCODE = b100 BYPASS = b111 EXTEST = b000 SAMPLE/PRELOAD = b010
As part of that standard, the following pins are used on each of the ETT1 Chip Set devices. jtag_tck, jtag_tdi, jtag_tdo, jtag_tms, jtag_trst_L The id codes used for the ETT1 Chip Set are as follows: Dataslice Enhanced Port Processor Crossbar Scheduler 14367049h 14581049h 14372049h 14374049h
B.2
RESERVED MANUFACTURING TEST PINS
Each device in the ETT1 Chip Set has a number of manufacturing test pins. These pins are reserved for use during the manufacturing process. For functional operation, input test pins must be driven by a voltage source, while output test pins should be left unconnected. Depending on conventions used during PCB assembly, test inputs can be tied with a 1k ohm resistor to the appropriate VDD or GND supply. Such a convention allows for onboard test operation for unusual diagnostic purposes. Test outputs should still be soldered to the PCB for proper thermal and mechanical operation, and these outputs are often brought to the solder (bottom) side of the board through vias for standard test fixtures. For correct functional operation of the ETT1 devices, the following test inputs should be tied to VDD through a resistor: lssd_ce1_a, lssd_ce1_b, lssd_ce1_c1, lssd_ce1_c2, lssd_ce1_c3,
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
323
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
test_ri, test_lt, lssd_scan_in[0:15], plltest_in The following test inputs should be tied to GND through a resistor: ce0_io, ce0_scan, ce0_test, ce0_tstm3, test_re The following test outputs should be left floating: lssd_scan_out[0:15], mon[0:15], plltest_out The following test inputs must be driven low during reset and high during operation. These can be connected to the same reset signal used for the pwrup_reset_in_L input. NOTE: These inputs can also driven low during board assembly tests to tristate all outputs. Testing of board continuity can done during ICT. test_di1, test_di2 The following test output could be left floating. Optionally, it may be used to monitor that the internal PLL has locked to the ref_clk/ref_clkn inputs by checking the signal level during ETT1 operation. plllock
B.3
POWER SUPPLY CONNECTIONS
All supply inputs labeled VDD must be driven by a common 2.5 V (nominal) supply. All supply inputs labeled GND must be tied to a common (0 V) ground. All pins labeled UNUSED must not be tied to a power or ground plane. The supply input labeled VDDA must be connected to the common 2.5 V supply through a noise isolation circuit. One possible method is shown in Figure 86.
324
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 86. Example Noise Isolation Circuit
NOTE: This distance must be kept as short as possible VDD
100 nH VDDA
0.1 uF
330 pF
All supply inputs labeled VDDQ must be driven by a common 1.6 V (nominal) supply. All pins labeled Vref must be connected to a common voltage reference plane that is 0.75 V and is derived from VDDQ.This voltage reference plane (Vref) must be supplied by a 115 ohm / 100 ohm voltage divider. One possible method is shown in Figure 87.
Figure 87. Example Voltage Divider
VDDQ
R1=115 Ohm
Vref
R2=100 Ohm
B.4
EXAMPLE POWER UP SEQUENCE
The power up sequence, shown in Figure 88, consists of three parts before the device is fully operational: * * voltage ramp-up hard reset
*
*
Note : pwruprst_L is a synchronous (to ref_clk and soc_in), active low signal
PLL reset
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
325
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
The voltage ramp-up time is the time needed for the power to the device to stabilize above 2.3 V.
Figure 88. Voltage Ramp-up
Voltage Ramp-up Reset PLL Reset
V (2.3 V)
t
1mS
10mS
Note: This example is design dependent Note: pwruprst_L is a synchronous, active low signal
326
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Appendix C Interfacing Details for ETT1
C.1 COMPENSATING FOR ROUND TRIP DELAY BETWEEN LINECARD AND ETT1(LCS)
A key benefit of the LCSTM protocol is the physical separation of the ETT1 fabric and the linecards. This separation introduces some latency in the communication channel between the linecard and ETT1. The credit mechanism used in LCS ensures that this latency does not adversely impact the throughput of the system. However, the linecard must compensate for the latency in the communication channel so that the credit mechanism can function correctly. This section describes the operations that the linecard should perform.
C.1.1
Preventing Underflow of the VOQ
The linecard must respond to grants from the ETT1Chip Set within a certain period of time in order to sustain maximum throughput. In this section we explain how much time the linecard has in which to respond to grants. Consider the case where the virtual output queue in the port card in Figure 89 is full, with 64 cells waiting to be forwarded to the appropriate output. Now, assume that the ETT1 Scheduler starts to forward those cells, one per cell time. When the first cell is forwarded, a grant is sent to the linecard, indicating that another cell can be sent to that output. If that cell does not arrive before the VOQ is empty, then the throughput for that flow cannot be equal to the link rate. Therefore, the new cell forwarded by the linecard must arrive at the EPP/DS before the VOQ is emptied. The round trip time for the whole process must be less than or equal to the number of cells in the VOQ; 64 cells for an OC-192c flow and 16 cells for an OC-48c flow. Figure 89 shows this time divided between the EPP/DS devices and "everything else". The EPP/DS uses 22 cells plus the programmed_token_delay. Refer to Section 3.4.2.39 "Internal Delay Matching Adjustments" on page 188. Subtract the total (22 cells + programmed_token_delay) from 64 to get the budget for "everything else" (OC-192c cells). "Everything else" includes any delay on the port card (Serdes), a mux device, the propagation delay of the signal down the fiber, the Serdes at the linecard and the linecard device delay. To calculate the time available to the linecard device, subtract the other elements from the "for everything else" budget. The fiber delay is approximately 15 cell times (7.5 cells in each direction), assuming a 70m link.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
327
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 89. The Time Available to the Linecard
ETT1 Port
Cell Delay DS Linecard
Grant
Delay
EPP
"Everything else" budget
EPP/DS budget
The time available to the linecard is a function of the round trip minus the time used by the EPP/DS devices.
C.2
THE 8B/10B INTERFACE TO THE DATASLICE
This section describes the external interface to the Dataslice. It explains how each physical link should be configured, how the links are combined to carry cell traffic, and what errors may occur and what impact those errors may have. The Dataslice uses 12 physical links in order to provide a 10 Gbit/s aggregate channel rate. The 72 byte cells carried across the channel are striped across the 12 individual links. Figure 90 shows this striping in one direction.
328
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 90. Cells are Striped Across the 12 Physical Links
Bytes 0-5 Bytes 6-11 Bytes 12-17 Bytes 18-23 Bytes 24-29 Bytes 30-35 Bytes 36-41 Bytes 42-47 Bytes 48-53 Bytes 54-59 Bytes 60-65 Bytes 66-71 1 cell 1 cell 1 cell 1 cell Dataslice Linecard
The link technology used is the same as that used by Gigabit Ethernet (GE), except that it operates at 1.5 Gbaud (150MHz) instead of 1.25 Gbaud (125MHz). While the Dataslice can be programmed to use any arbitrary 8b/10b code, the ETT1system has been focused on the use of the GE standard and thus the 8b/10b code developed originally by IBM. The following discussion assumes the use of standard Serdes components; however an alternative 8b/10b scheme could be used if suitable physical level components are available.
C.2.1
Link
Before looking at how the channel operates, we first consider a single link in its various aspects: physical interface, initialization, and errors. C.2.1.1 Link Physical Interface
Each logical Dataslice device connects to a Serdes device via 10-bit wide transmit and receive interfaces, operating at 150MHz in a source synchronous mode - i.e. the data is accompanied by a suitable clock. The Serdes transmits a serialized bit stream to a 1.5Gbaud VCSEL and receives a serialized bit stream from a 1.5Gbaud PIN diode. The ETT1 port card provides a 150MHz oscillator that is used as a reference clock for the Dataslice parallel bus transmitters. In the egress direction this reference clock is provided to the Dataslice which adjusts the phase of the reference so that it is aligned in mid-bit time of the transmitted word. In the ingress direction the Serdes device recovers a nominal 150MHz receive clock from the incoming serial stream and provides it to the Dataslice receiver. The Dataslice expects to sample the incoming 10-bit word using the receive clock; the data is then re-synchronized internally within the Dataslice. Figure 91 illustrates the two directions.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
329
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 91. The 10-bit Data Paths
fotx_clk_in (~150MHz)
VCSEL
Serdes Tx
10
fotx0_txd[]
fotx_clk_out (~150MHz) VCSEL Serdes Tx 10 fotx1_txd[]
Dataslice Tx
fotx_txd[9:0] fotx_clk_out
forx0_clk (~150MHz) PIN diode Serdes Rx 10 forx0_rxd[] Dataslice Rx
forx1_clk (~150MHz) PIN diode Serdes Rx 10 forx1_rxd[]
forx_rxd[9:0] forx_clk_in
C.2.1.2
Link Initialization
At power-on, every individual link needs to be initialized so that the receiver can lock onto the incoming stream and can identify both word and cell boundaries. To this end, the transmitter must send a particular sequence of 8b/10b control words. Now, the Dataslice assumes that it will be supplied with a word clock (forx_clk) and so only needs to determine a cell boundary. Remember that each Dataslice only sees six words of each cell. So the Dataslice looks for one occurrence of control word 0 followed by five occurrences of some other control word (e.g. 1). The Dataslice has its own 10b->8b decoder and so it can map an arbitrary 8b/10b control word to its own decoded control word.
330
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Assuming the use of 8b/10b Serdes, then we suggest the following mapping:
Table 73. Suggested 8b/10b Decode Map Within the Dataslice
8b/10b code
K28.5 K27.7 K29.7
Dataslice code
Control 0 Control 1 through 254 Control 255
Figure 92. Initial Sequence Expected at Port Card
K28.5 Linecard
K27.7
K27.7
K27.7
K27.7
K27.7
K28.5 Serdes Dataslice
The initial control word sequence decoded by the Dataslice is expected to be {0,1,1,1,1,1,0,1,1...}
Once the Dataslice has seen five consecutive occurrences of the expected sequence it then locks its cell timing so that the K28.5 word corresponds to the first word in the cell. The linecard may instead send {K28.5, K29.7, K29.7, K29.7, K29.7, K29.7}. This simply tells the Dataslice that the linecard's receiver has locked onto the Dataslice's transmitted signal. This is useful because it tells the ETT1 CPU that the link is up and locked in both directions. The transmit side of the Dataslice sends the same sequence: it starts by sending {K28.5, K27.7, K27.7, K27.7, K27.7, K27.7}. When the receive side of the Dataslice has achieved cell frame lock then the transmit side changes to send {K28.5, K29.7, K29.7, K29.7, K29.7, K29.7} instead. The sequence {K28.5, K27.7, K27.7, K27.7, K27.7, K27.7} is called Idle-not-ready. The sequence {K28.5, K29.7, K29.7, K29.7, K29.7, K29.7} is called Idle-ready. Table 74 shows the state machine operated by the transmit side of the Dataslice.
Table 74. Dataslice Egress Truth Table
8B/10B Encoder Enable
0 1 1 1 1
Output Fifo Status
don't care 0 (FIFO empty) 0 1 (FIFO not empty) 1
8B/10B Ingress Link Ready
don't care 0 1 don't care don't care
10B Transmit Data (TxD)
Null (10'b0) K28.5, K27.7 K28.5, K29.7 Dxx.x Dxx.x
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
331
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
If the encoder is not enabled then all zeros are sent. If the Dataslice output FIFO is empty then the Dataslice sends either Idle-ready or Idle-not-ready, depending on the state of the receiver. Otherwise the Dataslice will send the data cell at the head of its transmit FIFO. Why K28.5, K27.7 etc? The Dataslice doesn't care what the actual 10b control words are. We have chosen K28.5 as it is the 8b/10b comma character which the Serdes receiver will try to lock to in order to determine the word boundary in the bit stream. The choice of K27.7 and K29.7 is somewhat arbitrary but they comply with the 8b/10b transmission rules. (Indeed any control characters can be used provided that the Dataslice decoder table is configured appropriately.) To summarize, the steps required to bring up the ingress and egress linecard to Dataslice links are: * * * Program the 8b/10b codec tables on each Dataslice; Transmit Idle-not-ready cells from the linecard to each Dataslice; Enable the 8b/10b codecs on each Dataslice causing the Dataslices to start sending Idle-not-ready cells to the linecard; Each Dataslice will transition to sending Idle-ready cells when it has acquired cell synchronization from the linecard; Likewise the linecard should transition to sending Idle-ready cells when it has acquired cell synchronization from each Dataslice.
*
*
Eventually both the linecard and the Dataslice will be receiving the Idle-ready sequence, indicating that both ends of the link have locked up. The linecard transmitter should then switch to sending data cells. The Dataslice has a receive FIFO which stores the incoming words only if they are decoded as a data word. Idle-not-ready and Idle-ready are not stored in the FIFO. A data cell is a sequence of six 8b/10b data words (Dxx.x). These data cells are stored in the receive FIFO. The Dataslice reads from the receive FIFO at a nominal 150MHz which is derived from a local oscillator. This local oscillator will operate at a slightly different frequency to that used on the linecard transmitter. To avoid overruns in the case where the linecard is at a slightly faster frequency than the Dataslice, the linecard must send Idle cells at regular periods. The period is determined by the maximum difference in frequency that is possible. Assuming +/-200 ppm oscillators, and allowing for storage of one extra cell in the FIFO, then 1 in 2,500 cells must be an Idle cell to avoid overrun. The oscillator on the ETT1 port card might be faster than that used on the linecard. Therefore, the Port Processor contains a register that will determine the period between Idle cells sent from the ETT1 port, to avoid overrunning the receive FIFO in the linecard.
332
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
C.2.1.3
Link Errors
If the link has no errors then the Dataslice will always see valid sequences of either six control words or six data words. In reality, errors will occur. The system designer must be aware of the implications of the different errors. * If a single word is corrupted to map to an illegal 10b word then the Dataslice will set DIR bit 23 and assert an interrupt signal to the OOB bus. If the error occurs at one of the Dataslices that deals with the LCS header or magic packets (i.e., Dataslices 0, 1, or 2) then the error is communicated to the Enhanced Port Processor which will ignore that cell. If a word is corrupted to look like another valid word, then this will not be detected, unless it corrupts the LCS header in which case the header CRC should detect it. If the user payload is corrupted then the cell will still be carried through the switch to the egress linecard. The Serdes recognizes the bit sequence 0011111xx as a comma character for byte alignment. This bit sequence occurs in the K28.5 comma character and it is used for link synchronization as described above. An error can potentially cause the 0011111xx bit sequence to appear anywhere in the serial bit stream, causing the Serdes to re-align to the corrupted 0011111xx comma and thus lose byte synchronization. This condition will cause a sequence of 8B/10B errors to be encountered by the 8B/10B decoder and the Dataslice will raise the DIR23 decoder error interrupt. The Serdes should re-acquire lock when the linecard next sends an Idle cell. To prevent fiber optics bit errors from throwing the Serdes out of byte synchronization the Serdes byte sync enable pin can be connected to a pin on the Dataslice that is accessible through the OOB. Once the Serdes has locked onto the right byte boundary software can make sure that the Serdes will no longer look for the 0011111xx comma character. A subtle yet serious error can occur if an idle cell is corrupted in such a way that the comma character as programmed in the Dataslice 8B/10B decoder lookup table appears anywhere other than at the start of the cell or several data words in the same cell are corrupted into valid control words. This can cause the entire cell to appear as a cell of the other type (data -> Idle or Idle -> data). While improbable, this can have serious consequences that are discussed in the section on Channel errors.
*
*
*
C.2.2
Channel
A channel consists of 121links and thus 12 logical Dataslices, each slice dealing with six bytes of the 72 byte cell. C.2.2.1 Synchronization
Each of the 12 channels carry a part (6-bytes) of the same LCS cell. Although the 12 channels are all operating at the same frequency, their phase may be offset with respect to each other. In other words, each Dataslice may receive its 6-byte portion of the cell at a slightly different time from its neighboring Dataslices on the same switch port. So that the 12 parts of a cell remain intact, and are all switched at the same time,
1. The EPP can support fourteen logical slices (i.e. seven Dataslice chips)
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
333
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
we need to realign the output from the 12 channels. This re-alignment (or synchronization) is achieved as follows. The Dataslices provide a mechanism that guarantees that the slices of the same cell are removed from the input FIFOs synchronously in the core ETT1 clock domain1. This synchronization is achieved by only removing cells from the input FIFO when all 12 Dataslices have received their 6 bytes of a given cell. To accomplish this, upon each new cell arrival, a token is passed from Dataslice 0 to the EPP, which in turn broadcasts the token back to every Dataslice after an additional programmed token delay. . Refer to Section 3.4.2.39 "Internal Delay Matching Adjustments" on page 188. Each token that is granted by the EPP allows for one cell to be removed from the input FIFO of each Dataslice and sent to the EPP. The token delay mechanism compensates for a certain amount of skew/delay measured from arrival at Dataslice 0 to arrival at Dataslices 1 through 11. Additional depth in the input FIFO, beyond the depth required to support the token delay, compensates for skew from arrival at Dataslices 1 through 11 to arrival at Dataslice 0. Table 75 shows the allowable skew in each direction for each possible value of the programmed token delay, in units of 150MHz clock cycle times.
Table 75. Programmed Token Delay vs. Inter-link Skew
Programmed Token Delay
0 1 2 3
Max Skew DS0 to DS1 through 11 (150 MHz Clock Cycles)
11 17 23 29
Max Skew DS1 through 11 to DS0 (150 MHz Clock Cycles)
19 13 7 1
Possible sources of skew include linecard launch timing, fiber, connector, serializer/deserializer, mux chip. The worst-case skew depends on the system design. Therefore the system designer must determine the worst-case skew in each direction relative to Dataslice 0, and select a programmed token delay value that will provide adequate protection against both worst cases. The smallest sufficient token delay should be chosen, because the programmed token delay adds to the (E)PP/DS contribution to the RTT and decreases the budget for the rest of the system (see C.1.2). In the transmit direction from the Dataslice to the linecard there is no synchronization. Each Dataslice makes its own decision, at every cell time, as to whether or not it has a cell to send. An Idle cell is transmitted whenever the output FIFOs is empty. The Dataslice egress start-of-cell synchronization is determined independently by each Dataslice based on the asynchronous reset pulse. Further, the empty to non-empty transition of the output FIFOs can be skewed by up to one clock cycle in the 150MHz transmit clock domain. Therefore, it is possible for transmit cells to be skewed by up to one cell time or six 150MHz transmit clock cycles at the Dataslice egress, as shown in Figure 93. Consequently, the linecard must resynchronize the incoming slices to form the complete cell.
1. All ETT1 devices operate from the same core clock of 200MHz.
334
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 93. Cells May Be Skewed Between Dataslices
Dataslice x cell n-1 max Tx skew cell n
Dataslice y cell n cell n+1
Dataslice z cell n cell n+1
C.2.2.2
Channel Errors
Each link in the channel can, of course, suffer bit errors as discussed in the section on link errors above. However, the channel can incur a particular type of error which the system designer must be aware of. As mentioned above, it is possible for three or more data words in a data cell to be corrupted to look like control words. The Dataslice decides whether a cell is a data cell or Idle cell based on the majority of words seen. It is important to understand that each Dataslice decides independently whether a received cell is an idle cell or a data cell. This decision is based on a majority vote among the six characters received in a cell time: Kxx.x characters identify idle cells and Dxx.x characters identify data cells. It is very unlikely but nevertheless possible that a burst of bit errors on the fiber corrupt more than half of the characters in a cell time in such a way that an idle cell is mistaken for a data cell and vice-versa. Because idle cells are not written into the input FIFO of the Dataslice this condition causes every subsequent cell to be misaligned as it is read from the input FIFO and passed on to the port processor. There are cases where the corruption of control words is recognized by the Dataslice and signalled to the linecard by dropping the ingress ready status (DIR#9 and revert to sending idle-not-ready cells). In other cases, this condition is only recognized by the port processor when the LCS header is misaligned and in that case LCS CRC error interrupts will be raised. If the misalignment occurs in the cell payload, the linecard must recognize this condition either by checking CRCs on a cell or packet level or by means of a keep-alive watchdog traffic stream (for example, a loopback TDM connection). The recommended recovery actions from this rare condition require the port to be taken down and brought back up again.
C.3
MANAGING CELL RATES (SETTING THE IDLE COUNT REGISTER)
To avoid cell loss due to buffer overrun it is necessary that the linecard does not send cells at a rate that exceeds the ETT1 port cards ability to accept cells. Note: by buffer overrun we refer to the shallow, rate-matching FIFOs within the various devices; we do not mean the ingress or egress queues. The same is true for the ETT1 port card: it should not send cells at a rate that will cause cell loss at the linecard.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
335
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
The Enhanced Port Processor ensures this by periodically inserting an Idle cell in the egress cell stream (to the linecard). Clearly, if the linecard can accept a cell rate that is fundamentally greater than the maximum rate at which cells are sent, then there is no problem. In practice, however, the rate at which cells are transmitted and the rate at which cells can be accepted are determined by the clocks at each end. These clocks have some inherent imprecision, usually expressed as some number of parts per million. This means that the true rate at which cells can be transmitted or accepted might be slightly greater or smaller than the nominal rate. The remainder of this section is an example to explain this in more detail. Example: We assume that the ETT1 port and Linecard operate at a nominal frequency of 200MHz with an imprecision of +/-200ppm. One (OC-192c) cell time is eight clocks or 40ns, corresponding to a nominal rate of 25M cells/s +/-5000 cells/s. 1. Linecard to ETT1 Port. Assuming a slow clock at ETT1, then the ETT1 port may only be able to accept 24,995,000 cells/s. The linecard must not exceed this rate. So, if the linecard has maximum frequency clock (200.04MHz) then it could send 25,005,000 cells/s. So, to avoid overrunning the ETT1 port the linecard must send 10,000 Idle cells per second to bring its effective cell rate down to that of the ETT1 port. 2. ETT1 Port to Linecard. In this direction the worst case is where the ETT1 port has the highest frequency clock and the linecard has the lowest frequency. So the ETT1 port must insert 10,000 Idle cells per second. This would be done by programming the value 2,499 into the Idle Count Register in the Enhanced Port Processor. This causes the Enhanced Port Processor to insert one Idle cell for every 2,499 cells transmitted.
C.4
AVOIDING DELAYS DUE TO IDLE CELLS
Figure 94 shows a simple view of a linecard. A grant is received from the ETT1 port and the linecard must immediately respond with the corresponding cell body. However, if the Idle counter expires just at that moment, then the linecard cannot send the cell body; it must send an Idle cell instead. This behavior can cause a cell body to be delayed, thus diminishing the effective round-trip time available to the linecard, as well as introducing extra queuing into the linecard.
336
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 94. The Idle Counter Can Delay an Outbound Cell
Linecard
Idle Counter
To TT1
Linecard fetches cell from buffer
Grant received from TT1 port
Figure 95. The Idle Counter Can Delay an Outbound Cell
Linecard
Idle Counter
To ETT1
Linecard fetches cell from buffer
Grant received from ETT1 port
To avoid this, we suggest that the linecard implement the following protocol: 1. Whenever the Idle counter expires a flag F is set to 1 and the Idle counter starts again.
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
337
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
2. Whenever the linecard receives an LCS cell that does not have a grant and F = 1 then reset F to 0 and send an Idle cell. This very simple algorithm ensures that an Idle cell can be sent whenever the linecard does not have to respond to a grant. The only requirement here is that the ETT1 port must send a minimum rate of non-grant cells. This can be achieved in one of two ways: 1. Program the Idle Count register in the EPP to send Idles slightly more frequently than the linecard. 2. Program the NoGrant register in the EPP to restrict the rate of grants coming from the EPP. NOTE: The maximum number of cells between Idles sent by the linecard, if it follows this algorithm, is then set by the following equation: max_non_idle = linecard_idle_count + min(EPP_idle_count, EPP_NoGrant_count) This means that the just setting the Idle count at the linecard is not sufficient to ensure a minimum Idle cell rate; the rate at which non-grant cells are received must be included in the equation. The only further complication is that the linecard also sends out requests for new cells. However, it is assumed that requests (new cells) arrive at the linecard at a slower rate than they can be forwarded to the ETT1 fabric due to speedup within the fabric. Given this, then the linecard just needs to provide a one-deep queue for outbound requests, as shown in Figure 96. Given the higher rate at which requests are sent, then the request FIFO will never overflow.
338
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
RELEASED Data Sheet PMC-2000164 ISSUE 3
PMC-Sierra, Inc.
PM9311/2/3/5 ETT1TM CHIP SET
ENHANCED TT1TM SWITCH FABRIC
Figure 96. One-deep Queue for Outbound Requests
Linecard
Cell arrives from upstream switch, generating a request
Idle Counter
To TT1
Linecard fetches cell from buffer
Grant received from TT1 port
PROPRIETARY AND CONFIDENTIAL TO PMC-SIERRA, INC., AND FOR ITS CUSTOMERS' INTERNAL USE
339

▲Up To Search▲

Price & Availability of PM9311-UC

	To Download PM9311-UC Datasheet File
If you can't view the Datasheet, Please click here to try to view without PDF Reader .